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Preface 


Point processes underlie a range of activities within the human body. Neural spiking, 
rhythmic cardiac contraction, and pulsatile hormone secretion all have binary-like 
mechanisms at their core. The field of estimating latent states tied to point process 
observations has seen a steady growth over a period that has now exceeded two 
decades. These state estimation methods have found applications across a range of 
specialities including behavioral learning, brain-computer interfaces, sleep studies, 
heart rate variability analysis, anesthesia, endocrinology, and human emotion. The 
field has also seen an expansion from some of the early state-space estimators 
that were limited to point process observations alone to mixed estimators that can 
incorporate both binary and continuous-valued observations. 

Despite the growth in the field and its widespread applicability, several chal- 
lenges are encountered by those with an undergraduate engineering degree who 
wish to begin developing these types of estimators. While the estimators are similar 
to regular Kalman filters, their design is not typically approached in the way that 
regular Kalman filters are. Instead, the design of state estimators for point process 
observations is usually approached from a statistical Bayesian point-of-view, rather 
than from the typical least squares minimization perspective. Moreover, while a 
number of works can be found in the literature involving these types of point 
process Bayesian filters, there is no tutorial-like introduction to aid the beginner. 
Consequently, the student who wishes to begin research has to spend considerable 
time to learn the basics of point process Bayesian filter design; and that often 
from material in research papers which are not intended as tutorials. This book 
is an attempt to bridge the gap. Hence, an intended reader wishing to learn filter 
design is expected to have taken an undergraduate course in basic probability and 
statistics as well as a course in signals and systems. Some background in computer 
programming would also be necessary to implement the filters. A course in control 
systems, although not required, would be helpful as well. A reader merely intending 
to use the filters presented here, however, would not need this background but only 
require some basic proficiency with MATLAB. 

Point process state estimators have found a number of applications in fields 
related to physiology and medicine, some of which have been listed above. This 
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is partly due to the prevalence of point processes phenomena in different types of 
physiological signals. Successful collaborative research has also resulted based on 
drawing connections between state-space estimation and physiology—connections 
that serve to bridge the gap between engineering and medicine. Our book is also 
intended to serve the non-engineering community. Thus, the practitioner who does 
not wish to delve into all the mathematical detail underlying filter design but merely 
wishes to apply the tools can also use the book. Precisely for this purpose, the 
book is accompanied by a MATLAB toolbox of code examples that cover the 
different filters. Brief descriptions of the code are also provided at the end of the 
main chapters. It is our expectation that the endocrinologist, psychologist, or other 
researcher who wishes to estimate latent physiological states underlying binary and 
continuous-valued measurements would thus benefit as well. 

Point processes are everywhere, if you look for them. Earthquakes, crime 
incidents, rainfall, disease infections, customer arrivals at a bank, and website 
visits can all be modeled using point processes. Therefore, the applicability of this 
book isn't solely limited to physiology. Researchers in agriculture, epidemiology, 
climatology, etc. whose work involves point process phenomena can all find the 
book a helpful aid. 

The ever-increasing role of technology in our lives is undeniable. We rely 
on technology from everything to flying across continents to buying candy from 
a vending machine. It is not unlikely that the foreseeable future will involve 
an explosion of smart electronics everywhere. These networked devices will be 
connected to the cloud where more powerful analytical tools sift through the raw 
data. As with the case of latent variable estimation using state-space models, the 
merging between physical components and cyber analytics will seek to extract 
underlying or hidden patterns and information from the sensed data. The same 
scenario is also applicable to the human body. It is likely that the future will involve 
an increased adoption of bioelectric and biochemical sensors that will play a crucial 
role in our well-being. The sensed physiological signals will inevitably contain some 
point process data, all of which capture information regarding latent states within 
the human body and brain. Thus, with the aid of appropriate mathematical tools, 
some of which are covered in this book, a sensor-laden wristwatch in the future may 
be able to precisely tell you some of what's happening inside your brain and your 
body—an idea that led to an Innovators Under 35 recognition by MIT Technology 
Review for one of the authors. 

It is our hope that this book will be of help to students, researchers, and 
practitioners alike. 


Colombo, Sri Lanka Dilranjan S. Wickramasuriya 
New York, NY, USA Rose T. Faghih 
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Chapter 1 Y 
Introduction FEEN 


The human body is an intricate network of multiple functioning sub-systems. Many 
unobserved processes quietly keep running within the body even while we remain 
largely unconscious of them. For decades, scientists have sought to understand how 
different physiological systems work and how they can be mathematically modeled. 
Mathematical models of biological systems provide key scientific insights and also 
help guide the development of technologies for treating disorders when proper 
functioning no longer occurs. One of the challenges encountered with physiological 
systems is that, in a number of instances, the quantities we are interested in are 
difficult to observe directly or remain completely inaccessible. This could be either 
because they are located deep within the body or simply because they are more 
abstract (e.g., emotion). Consider the heart, for instance. The left ventricle pumps 
out blood through the aorta to the rest of the body. Blood pressure inside the aorta 
(known as central aortic pressure) has been considered a useful predictor of the 
future risk of developing cardiovascular disease, perhaps even more useful than the 
conventional blood pressure measurements taken from the upper arm [1]. However, 
measuring blood pressure inside the aorta is difficult. Consequently, researchers 
have had to rely on developing mathematical models with which to estimate 
central aortic pressure using other peripheral measurements (e.g., [2]). The same 
could be said regarding the recovery of CRH (corticotropin-releasing hormone) 
secretion timings within the hypothalamus—a largely inaccessible structure deep 
within the brain—using cortisol measurements in the blood based on mathematical 
relationships [3]. Emotions could also be placed in this same category. They are 
difficult to measure because of their inherently abstract nature. Emotions, however, 
do cause changes in heart rate, sweating, and blood pressure that can be measured 
and with which someone’s feelings can be estimated. What we have described so 
far, in a sense, captures the big picture underlying this book. We have physiological 
quantities that are difficult to observe directly, we have measurements that are easier 
to acquire, and we have the ability to build mathematical models to estimate those 
inaccessible quantities. 
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Let us now consider some examples where the quantities we are interested in 
are rather abstract. Consider a situation where new employees at an organization 
are being taught a new task to be performed at a computer. Let us assume that 
each employee has a cognitive “task learning” state. Suppose also that the training 
sessions are accompanied by short quizzes at the end of each section. If we were 
to record how the employees performed (e.g., how many answers they got correct 
and how much time they took), could we somehow determine this cognitive learning 
state, and see how it gradually changes over time? The answer indeed is yes, with the 
help of a mathematical model, we can estimate such a state and track an employee's 
progress over time. We will, however, first need to build such a model that relates 
learning to quiz performance. As you can see, the basic idea of building models that 
relate difficult-to-access quantities to measurements that we can acquire more easily 
and then estimate those quantities is a powerful concept. In this book, we will see 
how state-space models can be used to relate physiological/behavioral variables to 
experimental measurements. 

State-space modeling is a mature field within controls engineering. In this book, 
we will address a specific subset of state-space models. Namely, we will consider 
a class of models where all or part of the observations are binary. You may wonder 
why binary observations are so important? In reality, a number of phenomena 
within the human body are binary in nature. For instance, the millions of neurons 
within our bodies function in a binary-like manner. When these neurons receive 
inputs, they either fire or they do not. The pumping action of the heart can also be 
seen as a binary mechanism. The heart is either in contraction and pumping out 
blood or it is not. The secretion of a number of pulsatile hormones can also be 
viewed in a similar manner. The glands responsible for pulsatile secretion are either 
secreting the hormone or not. In reality, a number of other binary phenomena exist 
and are often encountered in biomedical applications. Consequently, physiological 
state-space models involving binary-valued observations have found extensive 
applications across a number of fields including behavioral learning [4—9], position, 
and movement decoding based on neural spiking observations [10-17], anesthesia, 
and comatose state regulation [18—20], sleep studies [21], heart rate analysis 
[22, 23], and cognitive flexibility [9, 24]. In this book, we will see how some of 
these models can be built and how they can be used to estimate unobserved states of 
interest. 


1.1 Physiology, State-Space Models, and Estimation 


As we have just stated, many things happen inside the human body, even while 
we are largely unaware that they are occurring. Energy continues to be produced 
through the actions of hormones and biochemicals, changes in emotion occur 
within the brain, and mental concentration varies throughout the day depending 
on the task at hand. Despite the fact that they cannot be observed, these internal 
processes do give rise to changes in different physiological phenomena that can 
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indeed be measured. For instance, while energy production cannot be observed 
directly, we can indeed measure the hormone concentrations in the blood that affect 
the production mechanisms. Similarly, we can also measure physiological changes 
that emotions cause (e.g., changes in heart rate). Concentration or cognitive load 
also cannot be observed, but we can measure how quickly someone is getting their 
work done and how accurately they are performing. Let us now consider how these 
state-space models relate unobserved quantities to observed measurements. 

Think of any control system such as a spring-mass-damper system or RLC 
circuit (Fig. 1.1). Typically, in such a system, we have several internal state variables 
and some sensor measurements. Not all the states can be observed directly. However, 
sensor readings can and do provide some information about them. By deriving 
mathematical relationships between the sensor readings and the internal states, we 
can develop tools that enable us to estimate the unobserved states over time. For 
instance, we may not be able to directly measure all the voltages and currents in 
a circuit, but we can use Kirchoff’s laws to derive relationships between what we 
cannot observe and what we do measure. Similarly, we may not be able to measure 
all the positions, velocities, or accelerations within a mechanical system, but we 
can derive similar relationships using Newton’s laws. Thus, a typical engineering 
system can be characterized via a state-space formulation as shown below (for the 
time-being, we will ignore any noise terms and non-linearities). 


Xk41 = AX, + Buy (1.1) 
Ve = CX. (1.2) 


Here, x, is a vector representing the internal states of the system, yg is a vector 
representing the sensor measurements, uz is an external input, and A, B, and C 
are matrices. The state evolves with time following the mathematical relationship 
in (1.1). While we may be unable to observe x; directly, we do have the sensor 
readings yz that are related to it. The question is, can we now apply this formulation 
to the human body? In this case, xy could be any of the unobserved quantities we 
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Fig. 1.1 Some examples of engineering systems that can be modeled using state-space represen- 
tations. The left sub-figure depicts a spring-mass-damper system, and the right sub-figure depicts 
an RLC circuit. We may not be able to directly observe all the states within each system, but we 
can build state-space models and use whatever measurements we have to estimate them 
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just mentioned (e.g., energy production, emotion, or concentration) and yz could be 
any related physiological measurement(s). 

In this book, we will make use of an approach known as expectation- 
maximization (EM) for estimating unobserved quantities using state-space models. 
In a very simple way, here is what the EM algorithm does when applied to state 
estimation. Look back at (1.1) and (1.2). Now assume that this formulation governs 
how emotional states (xg) vary within the brain and how they give rise to changes 
in heart rate and sweat secretions (yz) that can be measured. We do not know x, for 
k — 1,2,..., K, and neither do we know A, B, or C. We only have the recorded 
sensor measurements (features) yz. First, we will assume some values for A, B, and 
C, i.e., we will begin by assuming that we know them. We will use this knowledge 
of A, B, and C to estimate xy fork = 1,2,..., K. We now know x; at every point 
in time. We will then use these x;’s to come up with an estimate for A, B, and 
C. We will then use those new values of A, B, and C to calculate an even better 
estimate for xz. The newest x will again be used to determine an even better A, B, 
and C. We will repeat these steps in turn until there is hardly any change in xx, A, 
B, or C. Our EM algorithm is said to have converged at this point. The step where 
X, is estimated is known as the expectation-step or E-step and the step where A, B, 
and C are calculated is known as the maximization-step or M-step. For the purpose 
of this book, we will label the E-step as the state estimation step and the M-step as 
the parameter estimation step. What follows next is a basic description of what we 
do at these steps in slightly more detail. 


1.1.1 State Estimation Step 


As we have just stated, our EM algorithm consists of two steps: the state estimation 
step and the parameter estimation step. At the state estimation step we assume 
to know A, B, and C and try to estimate x, fork = 1,2,..., K. We do this 
sequentially. Again, look back at (1.1) and (1.2). Suppose you are at time index 
k and you know what A, B, C, and xz-¡ are, could you come up with a guess 
for xy? You can also assume that you know what the external input uz is for 
k = 1,2,..., K. How would you do determine xz? First, note that we can re-write 
the equations as 


Xy = AXx-1 + Buz-1 (1.3) 
Ve = CXk. (1.4) 


If you knew A, B, C, xx-1, and uj, and had to determine x; just at time index 
k, you would encounter a small problem here. Do you see that x; appears in both 
equations? You could simply plug-in the values of x,_; and uj; into (1.3) and 
get a value for xz. Since you are using the past values up to time index (k — 1) to 
determine x;, this could be called the predict step. You are done, right? Not quite. If 
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you determine x; solely based on (1.3), you would always be discounting the sensor 
measurement y; in (1.4). This sensor measurement is also an important source of 
information about xg. Therefore, at each time index k, we will first have the predict 
step where we make use of (1.3) to guess what x; is, and then apply an update step, 
where we will make use of y; to improve the x; value that we just predicted. The 
full state estimation step will therefore consist of a series of repeated predict, update, 
predict, update, ... steps fork = 1,2,..., K. At the end of the state estimation step, 
we will have a complete set of values for xx. 

Dealing with uncertainty is a reality with any engineering system model. These 
uncertainties arise due to noise in our sensor measurements, models that are unable 
to fully account for actual physical systems and so on. We need to deal with this 
notion of uncertainty when designing state estimators. To do so, we will need some 
basic concepts in probability and statistics. What we have said so far regarding 
estimating xy can be mathematically formulated in terms of two fundamental ideas 
in statistics: mean and variance. In reality, (1.3) and (1.4) should be 


Xy = AXx-1 + Buy, + e (1.5) 
yx = CXk + Vk, (1.6) 


where e; is what we refer to as process noise and vx is sensor noise. Therefore, when 
we “guess” what x; is at the predict step, what we are really doing is determining the 
mean value of x; given that we have observed all the data up to time index (k — 1). 
There will also be a certain amount of uncertainty regarding this prediction for xx. 
We quantify this uncertainty in terms of variance. Thus we need to determine the 
mean and variance of x; at our predict step. But what happens after we observe yg? 
Again, the idea is the same. Now that we have two sources of information regarding 
Xy (one based on the prediction from xx-1 and ux—1, and the other based on the 
sensor reading yx), we will still be determining the mean and variance of xz. So we 
need to calculate one mean and variance of x; at the predict step, and another mean 
and variance of x, at the update step. 


1.1.2 Parameter Estimation Step 


Recall that our EM algorithm iterates between the state estimation step and 
the parameter estimation step until convergence. Assume that we sequentially 
progressed through repeated predict, update, predict, update, ... steps for k = 
1, 2, ..., K and determined a set of mean and variance (uncertainty) values for xx. 
How could we use all of these mean and variance values to determine what A, B, 
and C are? Here is how we proceed. We first calculate the joint probability for all 
the x; and yz values. The best estimates for A, B, and C are the values that maximize 
this probability (or the log of this probability). Therefore, we need to maximize this 
probability with respect to A, B, and C. One simple way to determine the value at 
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which a function is maximized is to take its derivative and solve for the location 
where it is 0. This is basically what we do to determine A, B, and C (in reality, we 
actually maximize the expected value or mean of the joint log probability of all the 
x; and yz values to determine A, B, and C). 


1.1.3 Algorithm Summary 


In summary, we have to calculate means and variances at the state estimation 
step and derivatives at the parameter estimation step. We will show how these 
equations are derived in a number of examples in the chapters that follow. The 
EM approach enables us to build powerful state estimators that can determine 
internal physiological quantities that are only accessible through a set of sensor 
measurements. 

What we have described so far is a very simple introduction to the EM algorithm 
as applied to state estimation. Moreover, for someone already familiar with state- 
space models, the predict and update steps we have just described should also sound 
familiar. These are concepts that are found in Kalman filtering. The derivation of the 
Kalman filter equations is generally approached from the point of view of solving 
a set of simultaneous equations when new sensor measurements keep coming in. In 
this book, we will not approach the design of the filters through traditional recursive 
least squares minimization approaches involving matrix computations. Instead, we 
will proceed from a statistical viewpoint building up from the basics of mean and 
variance. Nevertheless, we will use the terminology of a filter when deriving the 
state estimation step equations. For reasons that will become clearer as we proceed, 
we can refer to these state estimators as Bayesian filters. 


1.2 Book Outline 


State-space models have been very useful in a number of physiological applications. 
In this book, we consider state-space models that give rise, fully or partially, to 
binary observations. We will begin our discussion of how to build Bayesian filters 
for physiological state estimation starting with the simplest cases. We will start by 
considering a scalar-valued state x, that follows the simple random walk 


Xk = Xk-1 + £k, (1.7) 


where £y ~ N(O, 02) 1s process noise. We will consider how to derive the state 
and parameter estimation step equations when x gives rise to a single binary 
observation nz. We will next proceed to more complicated cases. For instance, one 
of the cases will be where we have a forgetting factor p such that 
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Xk = PXk-1 + €k, (1.8) 


and x, gives rise to both a binary observation ng and a continuous observation rx. 
An even more complicated case will involve an external input so that 


Xk = PXk-1 + aly + £k, (1.9) 


where o7, is similar to the Bu, in (1.1), and x; gives rise to a binary observation 
nx and two continuous observations rg and sg. As we shall see, changes in the state 
equation primarily affect the predict step within the state estimation step. In contrast, 
changes in the observations mainly affect the update step. 

Note that we mentioned the observation of binary and continuous features. When 
introducing the concept of physiological state estimation for the first time, we used 
the formulation 


Ye = Cx + vi (1.10) 


for the sensor measurements. In reality, this represents a very simple case, and 
the equations turn out to be similar to that of a Kalman filter. Sensor measure- 
ments in biomedical experiments can take many forms. They can take the form 
of binary-valued observations, continuous-valued observations, and spiking-type 
observations, to name a few. For instance, we may need to estimate the learning 
state of a macaque monkey in a behavioral experiment based on whether the monkey 
gets the answers correct or incorrect in different trials (a binary observation), 
how quickly the monkey responds in each trial (a continuous observation), and 
how electrical activity from a specific neuron varies over the trials (a spiking-type 
observation). These types of measurements result in filter equations that are more 
complicated than in the case of a Kalman filter. We will rely heavily on Bayes’ rule 
to derive the mean and variance of x, at the update step in each case. 

While the state estimation step relies primarily on mean and variance calcula- 
tions, the parameter estimation step relies mainly on derivatives. At the parameter 
estimation step, we take the derivatives of the probability terms (or equivalently, of 
the log-likelihood terms) to determine the model parameters. For instance, if we use 
the state equation in (1.8), we will need to derive p at the parameter estimation 
step. Moreover, we also need to determine the model parameters related to our 
observations. For instance, we may choose to model a continuous observation rg 
as 


rk = yo + yAXk + Uk, (1.11) 


where yo and yı are constant coefficients and vy ~ N(0, 0?) is sensor noise. The 
three parameters yo, y1, and o2 all need to be determined at the parameter estimation 
step. We could thus divide the parameter estimation step derivations into two parts. 
First, there will be the derivations for model parameters in the state equation (e.g., 


8 1 Introduction 


p, a, and 071 And second, there will be the derivations corresponding to each of 
the observations (features). Choosing to include a continuous-valued observation 
in a state-space model will necessitate the determination of a certain set of model 
parameters. Adding a spiking-type observation necessitates a further set of model 
parameters. We will see examples of these in due course. 

Having laid some of the basic groundwork, we will next proceed with our tutorial 
discussion of how to derive the state and parameter estimation step equations for 
several different physiological state-space models. Shown below is a list of the state- 
space models we will look at along with examples of where they have been applied: 


* State-space model with one binary observation: 


— Behavioral learning [4] 
— Sympathetic arousal estimation using skin conductance signals [25, 26] 


* State-space model with one binary and one continuous observation: 


— Behavioral learning [5] 

— Emotional valence estimation using electromyography (EMG) signals [27] 

— Seizure state estimation using scalp electroencephalography (EEG) signals 
[28] 


* State-space model with one binary and two continuous observations: 


— Sympathetic arousal estimation using skin conductance signals [29] 
— Energy state estimation using blood cortisol concentrations [30] 


* State-space model with one binary, two continuous, and a spiking-type observa- 
tion: 


— Sympathetic arousal estimation using skin conductance and electrocardiogra- 
phy (EKG) signals [31] 


e State-space model with one marked point process (MPP) observation: 
— Sympathetic arousal estimation using skin conductance signals [32] 
* State-space model with one MPP and one continuous observation: 


— Energy state estimation using blood cortisol concentrations [33] 
— Sympathetic arousal estimation using skin conductance signals [33] 


Wearable and smart healthcare technologies are likely to play a key role in the 
future [34, 35]. A number of the state-space models listed above have applicability to 
healthcare. For instance, patients suffering from emotional disorders, hormone dys- 
regulation, or epileptic seizures could be fitted with wearable devices that implement 
some of the state-space models (and corresponding EM-based estimators) listed 
above for long-term care and monitoring. One of the advantages of the state-space 
framework is that it readily presents itself to the design of the closed-loop control 
necessary to correct deviation from healthy functioning. Consequently, state-space 
controllers can be designed to treat some of these disorders [36, 37]. Looking at the 
human body and brain from a control-theoretic perspective could also help design 
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bio-inspired controllers that are similar to its already built-in feedback control loops 
[38, 39]. The applications, however, are not just limited to healthcare monitoring, 
determining hidden psychological and cognitive states also has applications in fields 
such as neuromarketing [40], smart homes [41], and smart workplaces [42]. 


Excursus—A Brief Sketch of How the Kalman Filter Equations Can be 
Derived 
Here we provide a brief sketch of how the Kalman filter equations can 
be derived. We will utilize an approach known as recursive least squares. 
The symbols used within this excursus are self-contained and should not be 
confused with the standard terminology that is used throughout the rest of this 
book. 

Suppose we have a column vector of unknowns x and a column vector of 
measurements y, that are related to each other through 


yi = Aix t ei, (1.12) 
where A; is a matrix and ej ^ N(0, Xj) is noise (X, is the noise 
covariance matrix). In general, we may have more measurements than we 
have unknowns. Therefore, a solution to this system of equations is given by 


xy = (ATE LIA) [ATE ‘yn, (1.13) 


where we have used x; to denote that this solution is only based on the first 
set of measurements. Now suppose that we have another set of measurements 
yo such that 


Yo = Aox t €, (1.14) 
where Ap is a matrix and e2 ~ N (0, £2). In theory, we could just concatenate 
all the values to form a single set of equations and solve for x. However, this 
would result in a larger matrix inversion each time we get more data. Is there 
a better way? It turns out that we can use our previous solution x; to obtain a 


better estimate x2 without having to solve everything again. If we assume that 
e; and e» are uncorrelated with each other, the least squares solution is given 


by 
1 zi =l T =l T 
BGs) Ca aire) (A ee 
A2 0 5 Az A2 0 X» y2 
xl O aed Dl o T 
= T AT 1 T AT yı 
-[(1 43)( 0 N) (4 43)( 0 20 
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X2 = xj K AX + E — Pj AT (3o + ABAT AZ PI] TZ yo. 
(1.26) 


When multiplying the terms on the right, we will define the term O = (22 + 
A2 PLANT Making this substitution, we obtain 
xo = xi — KAyxi + (Pi BAI Qaz PI) ATE; y (1.27) 


Xo = x; — K Aox; + PA) X ly; — P, A] QAPI ATX! y2. (1.28) 


Here is where we will use a small trick. We will insert Q Q^! into the third 
term and then simplify. 


x; = xi - KA2x1 + PIAJOO” E) y — PAJQAS PAIX; yo 
(1.29) 


ES KA2xı + AT Q(Q! - AP, AD) z;! ya. (1.30) 


Since Q = (£2 + AP AT) !, Q^! = X2 + A? P A}. We will substitute this 
into (1.30) to obtain 


X2 = X] — KA + PAL O(X2 + A5 P1 AT = A; PAD)Z; ly (1.31) 
= x, — KA2x1 + PLA] QY2Y;! yo (1.32) 
= x; — KA + PA) Qy2. (1.33) 


Note that P AT Q = Pi A] (£2 + A2 P AT)! = K. Therefore, 


X2 = X1 — KA2x1 + Ky2 (1.34) 
= x; + K(y2 — A2x1). (1.35) 


What does the final equation mean? We simply take our previous solution x4, 
predict what y2 will be by multiplying it with A», calculate the prediction 
error yo — AoX;, and apply this correction to x; based on the multiplication 
factor K. These equations, therefore, provide a convenient way to continually 
update x when we keep receiving more and more data. 
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Excursus—A Brief Sketch of How the EM Algorithm Works 
Here we will provide a brief overview of how the EM algorithm works in 
the kind of state estimation problems that we shall see. Assume that we have 
a set of sensor measurements Y = (yj, yo,..., yx} and a set of unobserved 
states A = (x1, X2, ..., xx} that we need to estimate. We also have the model 
parameters © that need to be determined. 

Let us begin by asking the question as to how we can determine ©. 
In general, we select © such that it maximizes the probability p(O[). 
Assuming that we do not have a particular preference for any of the © values, 
we can use Bayes’ rule to instead select the © that maximizes p(Y|©). Now, 


p(Y|9) = [ pæ nyoa. (1.36) 


We do not know what the true © is, but let us make a guess that it is 6. Let us 
now introduce the term p(X | N ©) into (1.36). 


XIU nG 
pQle- / pene or A yor (137) 
x pex y nó) 
=| ae MENDA n (1.38) 
x pynó) 


Take a moment to look carefully at what the integral is doing. It is actually 
calculating the expected value of the fraction term with respect to p(4|YNO). 
Taking the log on both sides, we have 


^ p(X N VIO) 
log [ p(Y|O)] = lo [i ene aa (1.39) 
SAPIEN Ie jhe pony n) 
Since log(-) is a concave function, the following inequality holds true. 
^ (£ nyle) 
l RUIZ > f (X|YNO)1 E (1.40) 
og[pQ/l6)] BEEN BI eran GF 
log [p10)] = 1 P(X|YN ©) log [p(X N ald Y 
Lan gia [per n aa (141) 


Recall that we set out to choose the © that maximized p(Y|G), or that 
equivalently maximized log [p Q/Ie)]. Typically, we would approach this 
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maximization by calculating the derivative of the probability term with 
respective to O, set it to 0, and then solve. For instance, if we had a 
continuous-valued observation rg in our state-space model, we would have 
to take the derivatives with respect to yo, y1, and G set them each to 0, and 
solve. Look back at (1.41). Assume we were to calculate the derivative of the 
term on the right-hand side of the inequality with respective to ©. Do you 
see that the second term does not contain ©? In other words, the derivative 
would just treat the second term as a constant. If we had to determine yo, y1, 
and Ge, for instance, they would only be present in the first term when taking 
derivatives. We can, therefore, safely ignore the second term. This leads to 
an important conclusion. If we need to determine the model parameters © by 
maximizing log [ D(Y| 0)], we only need to concentrate on maximizing 


[eno o ©) tog [ow o vieexe. (1.42) 
We could equivalently write (1.42) as 


E y yo] log [px n y10)]| (1.43) 


since this is indeed an expected value. Do you now see the connection between 
what we have been discussing so far and the EM algorithm? In reality, what 
we are doing at the state estimation step is calculating E[V|V N 6]. At the 
parameter estimation step, we calculate the partial derivatives of the expected 
value of log [p(x n y/0)] with respect to all of the model parameters. 
During the actual implementation of the EM algorithm, we keep alternating 
between the two steps until the model parameters converge. At this point, we 


have reached one of the localized maximum values of E x yne| log [ p xn 


ie]. 
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Chapter 2 Y 
Some Useful Statistical Results heel or | 


The EM algorithms for state estimation that we consider in this book rely on basic 
concepts in statistics. In this chapter, we will review some results that will come 
in useful later on. Many of the concepts are introductory and only require a basic 
knowledge of probability and statistics. If you are already familiar with what is 
discussed here, feel free to skip ahead. 


2.1 Basic Concepts Related to Mean and Variance 


Shown below are some basic statistical results related to mean and variance that will 
be helpful when deriving the EM algorithm equations. 


Basic Statistical Results—Part A 

Given the random variables X; and Zx, and the constant values p and o, the 
following results hold true for mean and variance. We use E[-], V(-), and 
Cov(-) to denote the mean or expected value, the variance, and covariance, 
respectively. 


EX, + Zx] = E[X«] + E[Z;] (2.1) 
E[X; + 0] = E[X;] + 0 (2.2) 
E[pXx] = pE[X;] (2.3) 
V(Xx + Zk) = V (Xk) + V(Zx) + 2Cov(Xr, Zk) (2.4) 
(continued) 
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V(X +a) = V(Xx) (2.5) 
VLGX = pV (X). (2.6) 


2.2 Basic Statistical Results Required for Deriving the 
Update Equations in the State Estimation Step 


We will also require two results based on Bayes’ rule and Gaussian distributions, 
respectively, when deriving the update equations in the state estimation step. The 
two results are shown below: 


* Result 1: 


P(B|An C)P(A|C) 


P(AIBNC)= PEG 


We will consider the derivation of this result in two steps: 


— Step 1: 
P(ANBNC 
P(AIBNC) = "Esc (2.7) 
| P(ANBNC)_ PC) ds 
- P8nCO ^" PO) e 
P(ANBNC) 1 
= x (2.9) 
P(C) "C 
_ P(ANBIC) ats 
|». P(B|O) ` ` 
— Step 2: 
panpe oe (2.11) 
P(C) 
| P(AnBnO). PANC) Ei 
P(C) PANC) 
| P(AnBnO). PANC) T 


P(An C) P(C) 
P(BIANC)P(AIC). (2.14) 
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We will substitute the result for P(A N B|C) in (2.14) and put it into (2.10) to 
obtain 


P(ANBIC)  P(B|An C)P(A|C) 


PAIBNO= uo ^ PEO 


(2.15) 


Basic Statistical Results—Part B 
Letting A = Xx, B = Yg, and C = Y]:x-1, we can use the result just shown 
above to obtain 


P(Yi| Xy, Yi) PX Y 141) 


P(Xk Vik) = POXK|Yg, Yi:k-1) = 
P(Yk|Yia—1) 


(2.16) 


Recall that we split our state estimation into two steps: the predict step and 
the update step. At the predict step, we derive an estimate for x; given that we 
have not yet observed the sensor reading yg. This estimate is actually based on 
P(Xx|Y1:x-1) since information only available until time index (K — 1) is used 
to derive it. At the update step, we improve the predict step estimate based on 
P(Xx|Y1:x-1) to now include information from the new sensor measurement 
yr, 1.e., we make use of yz to obtain a new estimate based on P(X;|Y1.4). The 
result in (2.16) will come in very useful at the update step. 


* Result 2: 

The mean and variance of a Gaussian random variable can be obtained by 
taking the derivatives of the exponent term of its probability density function 
(PDF). 

Consider X ~ N (u, 0?). The PDF of X is given by 


Ary? 
e1” where q = COCHE 


2.17 
210? 20? — 


p(x) = 


To obtain the mean of X, we take the derivative of q and set it to 0 to determine 
where the maximum value occurs. 


dq | 20M) _ 
dx 20? 
=> ea (2.19) 


0 (2.18) 


Therefore, the mean value occurs at the location for x at which the derivative of 
the exponent term is equal to 0. 
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We next consider the variance. The second derivative of q with respect to x is 


dq -—1 
cen (2.20) 
And therefore, the variance is given by 
d 
zs is -(£3) Q.21) 


Basic Statistical Results—Part C 

In all the derivations of the state estimation step update equations, we will 
assume that the density functions are approximately Gaussian. We will also 
make use of what we have just shown: (i) the mean value is given by the 
location at which the first derivative of the exponent term is equal to 0; (ii) 
the variance is given by the negative inverse of the second derivative of the 
exponent term. 


2.3 General Observations Related to Gaussian Random 
Variables 


In general, for a set of independent Gaussian random variables Z; ^ N (ui, 07), 
the following holds true. 


X az; ~ N( Zam, Ys) (2.22) 


where the aj's are constant terms. Also, adding a constant term to a Gaussian random 
variable will cause it to remain Gaussian but have a shifted mean and unchanged 
variance. This can be verified from first principles (change of variables formula). 


Basic Statistical Results—Part D 
In general, for a set of independent Gaussian random variables Z; ~ 


N (ui » 071 


E aiZi ~ AL X dili, py se). (2.23) 
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Chapter 3 N 
State-Space Model with One Binary P 
Observation 


In this chapter, we will consider a state-space model where a single state variable x, 
gives rise to binary observations. We will see how the state and parameter estimation 
equations are derived for this case. However, prior to deriving any of the equations, 
we will first look at two example scenarios where the need for such a model arises. 

We human beings learn. We start learning since the time we were born, and 
learning continues thereafter as a life-long process. How exactly do we learn? 
And how do animals learn? These are interesting problems that scientists have 
investigated for years. One of the problems that arises in learning experiments 
with animal models is determining when an animal is considered to have learned 
something. For instance, suppose that a macaque monkey needs to learn how to 
correctly identify a particular visual target shown on a computer screen. The monkey 
may receive a reward for every correct answer. Similarly, a rat may have to learn to 
how to recognize an audio cue to receive a reward in a maze (Fig. 3.1). How could 
we know that the animal has actually learned? This is an interesting question. We 
could, for instance, come up with heuristic rules such as stating that the animal 
has indeed learned when five consecutive correct answers (or some other number) 
are recorded. But could something more systematic be developed? This problem 
is what motivated the work in [4]. Here, learning was characterized using a state- 
space model. Since correct and incorrect are the only possible trial outcomes, the 
observations are binary-valued. Moreover, rather than just deciding whether the 
animal has learned or not yet learned, the objective was to estimate a continuous 
learning state x, based on the sequence of binary responses ng. When learning 
has not yet occurred, more incorrect responses occur in the trials and x, remains 
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Correct/incorrect 
responses 


~ 7 Cognitive learning 
state 


Fig. 3.1 A rat in a T-maze experiment with binary-valued correct/incorrect responses. Binary- 
valued correct/incorrect responses can be used to estimate the cognitive learning state of a rat 
based on its responses in successive trials. The model was used in [4] for this purpose where the 
rat had to learn to recognize which direction to proceed in based on an audio cue 


low. However, as the animal begins to learn, more correct responses occur and 
xy increases. Thus it is possible to see how learning continuously progresses over 
successive trials. 

The second example relates to emotions and the nervous system. We primarily 
sweat to maintain internal body temperature. However, tiny bursts of sweat are also 
released in response to psychologically arousing stimuli. These variations in sweat 
secretions cause changes in the conductivity of the skin and can be picked up easily 
by skin conductance sensors. Since the sweat glands are innervated by nerve fibers 
belonging to the sympathetic branch of the autonomic nervous system [43], a skin 
conductance signal becomes a sensitive index of sympathetic arousal [44]. Now a 
skin conductance signal comprises a slow-varying tonic component on top of which 
a faster-varying phasic component is superimposed [45, 46]. The phasic component 
consists of what are known as skin conductance responses (SCRs). These SCRs 
have characteristic bi-exponential shapes. Each of these SCRs can be thought of as 
being produced by a single burst of neuroelectric activity to the sweat glands [47]. 
It is these phasic SCRs that give a skin conductance signal its “spikey” appearance 
(Fig. 3.2). A deconvolution algorithm can be used to recover the bursts of neural 
activity underlying a skin conductance signal [47—50]. Importantly, the occurrence 
of these neural impulses is related to a person's arousal level. In particular, the higher 
the underlying sympathetic arousal, the higher the rate at which neural impulses 
to the sweat glands (or SCRs) occur [51]. Thus the same state-space model with 
binary observations based on neural impulses to the sweat glands was used in [26] 
to estimate sympathetic arousal. By tracking the occurrence of the impulses ng, a 
person's arousal state could be estimated over time. 
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A Skin Conductance Signal 
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Fig. 3.2. A deconvolved skin conductance signal. A skin conductance signal comprises both a 
tonic and phasic component. The neural impulses underlying phasic variations can be extracted 
via deconvolution. The figure depicts a skin conductance signal (blue) and the sequence of neural 
impulses that underlie its phasic variations (red). From [32], used under Creative Commons CC- 
BY license 


3.1 Deriving the Predict Equations in the State Estimation 
Step 


Let us now consider the state-space model itself. For simplicity, we will also not 
use upper case letters for the unknowns although they are indeed random variables. 
Instead, we will follow the more familiar notation for state-space control systems 
with lower case letters. Let us begin by assuming that x; evolves with time following 
a random walk. 


Xk = Xk-1 + £k, (3.1) 


where the process noise term e; ~ AV (0, o2) is independent of any of the x, values. 

For now, let us not think of (3.1) as being the state equation in a control system. 
Instead, let us just consider (3.1) purely as a relationship between three random 
variables. Supposing we only had this equation and had to determine xg, what would 
be the best guess that we could come up with and how uncertain would we be about 
it? Our best estimate for x, would be its mean, and the uncertainty associated with 
it would be its variance. We will use the basic formulas in (2.1)-(2.6) to determine 
the mean and variance of xx. We will first derive the mean. 


Exe] = Elxx-1 + ex] (3.2) 
= Elxx-1] + Elex] using (2.1) (3.3) 
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= Elxx-1] since Riel = 0 (3.4) 


Elx] = Xk-11k-1> (3.5) 


where we have used the notation x,—1|x—1 to denote the expected value IE[x;....; ]. In 
a typical state-space control system, x,—1|x—1 represents the best estimate for xy... 
given that we have observed all the sensor measurements up to time index (k — 1). 
We will also use the notation IE[x;] = xx¡x-1 to denote the mean state estimate at 
time index k, given that we have only observed the sensor readings until time index 
(k — 1). 

Next we will derive the uncertainty or variance of x, using the same basic 
formulas. 


V (xk) = V (xk—1 + e) (3.6) 
= V (1-1) + V (ex) + 2Cov(xg-1, €k) using (2.4) (3.7) 
= V (x1-1) + V (ex) since ez is uncorrelated with any of the x, terms 

(3.8) 
^ VK) = og aya + 07, (3.9) 


where we have used the notation op. 111 fO denote the variance V (xy —1). Again, in 


a typical state-space control system, oi i.) represents the uncertainty or variance 
of xy. given that we have observed all the sensor readings up to time index (K — 1). 
Just like in the case of the mean, we will use the notation V (xg) = og g—1 to denote 
that this is the variance estimate at time index k, given that we have only observed 
the sensor readings until time index (k — 1). Therefore, our predict equations in the 
state estimation step are 


Xk|k—1 = Xk—1|k—1 (3.10) 


2 2 2 
Opi = 0111 Fe- (3.11) 


From our knowledge of Gaussian distributions in (2.23), we also know that xy is 
Gaussian distributed since x,_; and ez are Gaussian distributed and independent of 
each other. Since we have just derived the mean and variance of xg, we can state that 


: 2 
LKK "TRIK 11 


p(xk|ni:k-1) = e ii ; (3.12) 


1 
drad 


where the conditioning on 7r1,.4..; indicates that we have observed the sensor mea- 
surements up to time index (k — 1). What happens when we observe measurement 
ny at time index k? We will see how our estimates xjy-,; and ofri can be 
improved/updated once we observe n, in the next section. 
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When x; evolves with time following xy = xx-1 + €x, the predict equations 
in the state estimation step are 


Xk|k—1 = Xk—1|k—1 (3.13) 


2 2 2 
OQ] = Uijl + Fe - (3.14) 


3.2 Deriving the Update Equations in the State Estimation 
Step 


The binary observations nr that we consider here could be in the form of correct/in- 
correct responses in a behavioral experiment, neural impulses in a skin conductance 
signal, hormone pulses, etc. Let us assume that x, is related to the probability p; 
with which the binary events occur through 


1 


E 1 + e- (Potx)? (3-15) 


Pk 


where f is a constant. Here, py = P(ng = 1) and (1 — px) = P(ny = 0). 
Equation (3.15) depicts what is known as a sigmoid relationship. Accordingly, the 
higher x, is, the higher will be pz. In other words, the higher x, is, the higher the 
probability of 1’s occurring in the observations. 


At this point, we need to note an important result concerning the derivative of 
the sigmoid function. 


dpk =D 


= —(Bo+xx) pe 
deem o 010) 
1 e (Potx) ius 
7 1 + e (Bo+xx) d 1 + e- (Bots) [o 
1 1+ e (Boi) — 1 218 
7 1 + e (Botxx) e 1 + e- (Box) (EUR 
ee 1 Lee 3.19 
~ 1 + e (Botxx) l 1 + e- (Po--xx) oY 


DUL — px). (3.20) 


Il 
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Now the occurrence of ng = U or ny = 1 follows a Bernoulli distribution. 
Therefore, 
P(ne|xe) = py — pe) (3.21) 
1 nk 1 l—nk 
= £ + aa | | 1+ aa | : (5.22) 


We will also utilize another useful result here. For a positive number a, a = e!°8 
(this can be easily verified by taking the log value on both sides). We can use this to 
express DLL LYC) as shown below. 


poly) = pe = po^ oe 
— eog [rit a pot] (3.24) 
— gog |p ] oe [a7 507] (3.25) 
— ghilog(i)t nj) log(1- px) (3.26) 


l eke) +h 1- 
Sg" os (12) Hoe m (3.27) 


Now assume that we just observed ng. What would be our best estimate of x, given 
that we have observed 7.4? In other words, what is p(xy|n1.«), and how can we 
derive its mean and variance? We can use the result in (2.16) to determine what 
p(xxln1:x) is. 

p(nx|xx, ni-1i) poadnia-1) 


pini) = p(xk|nk, ni:k-1) = x (3.28) 
pGx|nia-1) 


Let us consider the terms in the numerator. Now p(ny|xy, ni4—1) = p(nglxx) since 
we have an explicit relationship between n, and x, as shown in (3.15), which makes 
the additional conditioning on the history 11.,—1 irrelevant. We know what p(nx|xx) 
is based on (3.26). We also know what p(xx|n1:x-1) is based on (3.12). 

We now need to determine the mean and variance of p(xx|n1:x). To do so, we 
will assume that it is approximately Gaussian distributed. Recall from the earlier 
results in (2.19) and (2.21) that the mean and variance of a Gaussian distribution 
can be derived from its exponent term alone. Therefore, we only need to consider 
the exponent of p(x&|ni..) and can ignore the other terms. We will therefore 
substitute the terms for p(nz|xx) and p(xy|n1:—1) in (3.26) and (3.12), respectively, 
into (3.28). 


2 
— (Ak XK 1) 
nelog( py) +A) log BG) x < pM 


pani) ex pu|xk) pGacdnia-1) oc e . 
(3.29) 
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Taking the log on both sides and labeling it as g, we have 


q = log[pGx|nis)] = nk log(pk) + (1 — ng) log — px) 


2 

— ee + constant. (3.30) 

204.1 

This equation provides us the exponent of p(xy[n1.4), which we will use to derive 

the mean and variance. We can obtain the mean by taking the first derivative of the 

exponent and then solving for the location where it is 0. Likewise the variance is 
given by the negative inverse of the second derivative. 

Let us first proceed with calculating the mean. We will make use of the formula 

for the derivative of pz in (3.20). 


dq 1 dpi 1 d 2(Xk — Xkjk-1) 
= Nk ny) (1 — px) 5 =0 
dxy Pr dxk (1 — px) dx 20]. 
(3.31) 
1 1 (Xk — Xklk—1) 
ny — pe(1 — pr) — (1 — ng) pid — pr) P -0 
Pk (1 — px) GOC) 
(3.32) 
(Xk — Xkik—1) 
nl — pr) — (1 — nk) pk HE =0 
Okik—1 
(3.33) 
(Xk — XkIk—1) 
nk — Pk EY 20 Q3) 
Okjk-1 
nope a) (3.35) 
Oklk—1 


Xk = KK + Oj k — Pk). (3.36) 


This equation gives us the mean of xg, which is now our new best estimate given 
that we have observed all the data up to time index k. We will call this new mean 
Xx\x- It is an improvement over x;|x—1, which did not include information from the 
latest observation. Since 


1 


Soar (3.37) 


Pk 


the x; term appears on both sides of (3.36). Therefore, the equation has to be solved 
numerically (e.g., using Newton's method). To make this dependency explicit, we 
will use the notation p; and express the mean as 
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Xklk = Xkk—1 + OK p(k — Dit), (3.38) 


We will next derive the variance. Now the first derivative of the exponent 
simplified to 


d (xk — Xk\k—-1) 
t= ng — pe a (3.39) 
dXi Okik-1 
The second derivative yields 
d?^q 1 
5 = -p(l — Pk) — =—. (3.40) 
ax; Oklk—1 


Based on our knowledge of how variance can be derived from the exponent term 
in a Gaussian distribution, the uncertainty or variance associated with our new state 
estimate is 


A) | e | 

2 

On, =— = 1— E 3.41 
klk p odia px (d — px) (3.41) 


Again, we will make the dependence of pz on xg explicit and state 


=ï 
1 
ofk = | ;— t pu (d — euo . (3.42) 
Okk—1 


When x; gives rise to a single binary observation ng, the update equations in 
the state estimation step are 


Xkjk = KET + Gg. L (k — Pu) (3.43) 
1 —1 
okk = Iz— + Pri — 2) 1 (3.44) 
Okik—1 


3.3 Smoothing in the State Estimation Step 


Although we previously stated that the state estimation step primarily consisted of 
the predict and update steps, in reality, there is a third step that we follow. The 
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equations for this third step, however, do not vary much depending on the state- 
space model and consequently do not require re-derivations every time we have a 
new model. In fact, as we shall see, there is only one case where we need to make 
changes to this third step. Now we first perform the predict, update, predict, update, 
... Steps in turn fork = 1,2,..., K to determine x, at each point in time. After 
coming to the end, we reverse direction and obtain a set of smoothened mean and 
variance estimates. The equations for this backward smoother are 


2 
O, 
apan M (3.45) 
OR Ak 
Xk|k = Xk|k + Åk(Xk+1|K — Xk+1/£) (3.46) 
2 2 2, 2 2 
Tx = Fike + ALC T Ikik) (3.47) 


The only change that occurs in these equations is if there is a forgetting factor p in 
the state equation (e.g., xx = pxx-1 + ex). In this case, we would have 


2 


o 
Ap & p— Ht (3.48) 

OR Ak 
Since we reverse direction making use of all the data through k = 1,2,..., K to 


obtain the smoothened mean and variance estimates, we use the notation Yk K and 
og x to denote their values. These new estimates turn out to be smoother since we 
now determine x, not just based on k = 1,2,...,k (what we have observed up to 
that point), but rather on k = 1,2, ..., K (all what we have observed). 

We will also make a further observation. We need to note that xx) y and o, y can 
be formally expressed as 


XK = Elxg|ni:x, O] (3.49) 


QU K = V(xInik, ©), (3.50) 


where © represents all the model parameters. In the case of our current state-space 
model, the only unknown model parameter is o2 (and fo, but we will assume that 
this is calculated differently). Why is the expected value conditioned on ©? Recall 
that the EM algorithm consists of the state and parameter estimation steps. At the 
state estimation step, we assume that we know all the model parameters and proceed 
with calculating xx. Mathematically, we could express this knowledge of the model 
parameters in terms of conditioning on ©. In reality, we could also have expressed 
Xk\k—1 and xy (and the variances) in a similar manner, i.e., 


Xgk-1 = Elxk|ni:k-1, O] (3.51) 
Xxx = Elxx|n1:x, 9]. (3.52) 
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Finally, we also need to note that we often require not only IE[xy |n: « , ©], but 
also LZ IHI K O] and Elxxxx+1|M1:k, ©] when we move on to the parameter 
estimation step. Making use of the state-space covariance algorithm [52], these 
values turn out to be 


igni, 0] = Uk = xz + AK (3.53) 


xx 1ln1:k, O] = Ue kt = Xk|KXk+1/K + ACL K (3.54) 


where we have defined the two new terms Ug and Uy 144. 


3.4 Deriving the Parameter Estimation Step Equations 


Recall our earliest discussion of the EM algorithm. To describe how it functioned, 
we assumed the simple state-space model 


XLI = AX; + Buy (3.55) 
yk = CXk. (3.56) 


We stated that, at our state estimation step, we would assume that we knew A, B, and 
C and then determine the best estimates for xg. The state estimation step consists of 
the predict step, the update step, and the smoothing step that we perform at then end. 
At the predict step, we make a prediction for x; using the state equation based on 
the past history of values. At the update step, we improve this prediction by making 
use of the sensor reading yz that we just observed. After proceeding through the 
predict, update, predict, update... steps, we finally reverse direction and perform 
smoothing. We primarily make use of the ideas of mean and variance at the state 
estimation step. It is after performing the state estimation step that we proceed to 
the parameter estimation step where we make use of the x, estimates and determine 
A, B, and C. We select A, B, and C to maximize a particular probability. This 
probability is the joint density of all our x, and yz values. We also stated that, in 
reality, it was not strictly the probability that we maximize, but rather the expected 
value or mean of its log. Do you now see why the state estimation step involved 
calculating the expected values of xg? 

Let us now consider the joint probability term whose expected value of the log 
we need to maximize. It is 


Pik N yk lO) = pk lxi, O) p(x1:k10). (3.57) 
Since we only observe a single binary variable, we have yy = nx. Therefore, 


pGyk ON1:K|O) = pu k|xik, O) p(x1:k10). (3.58) 
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We will first consider p(x1.« |©). What would be the total probability of all the x, 
values if we only knew the model parameters ©? In other words, if we had no sensor 
readings ng, what would be the probability of our x, values? To calculate this, we 
would only be able to make use of the state equation, but not the output equation. 
This probability is 


P(x1:K|O) = p(x110) x p(x2|x1, ©) 


x p(xal|xi, xa, ©) x ... x p(xk|xi, X2,..-,XK-1, O) (3.59) 


xp? 


K =P 
-IH x =e roe (3.60) 


Note that in the case of each term xg, xķ—1 contains within it the history needed to 
get to it. Let us take the log of this value and label it Q. 


- L (xk = S 
6 = > log ( 2102) -Y A L, (3.61) 
k=1 


Now the only model parameter we need to determine is o2 (ignoring fo). It turns 
out that o2 only shows up in this term involving p(xı:g |©) and not in the term 
involving p(ni:x |x1:« , ©). Let us now take the expected value of Q and label it Q. 


- K El xg — xx-1) 
Q = “E tog (222) = “ j 


(3.62) 
2 k=1 

What do we need to do at the parameter estimation step to determine o2? We simply 

need to take the derivative of Q with respect to o2, set it to O, and solve. But the 

expected value we need should be calculated conditioned on knowing O and having 

Observed n::x (i.e., we need E[x,|n1.« , O]). Do you now see why we expressed 

Xx k and SKK in the way that we did in (3.49) and (3.50)? 


3.4.1 Deriving the Process Noise Variance 


While it is possible to determine the starting state xo as a separate parameter, we 
follow one of the options in [4, 5] and set xo = x1. This permits some bias at the 
beginning. Therefore, 


s — Xk-1) d 


=K E 
Q= > log (2x07) 2 


k=2 


(3.63) 
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We will follow this method of setting x9 = x, in all of our parameter estimation 
step derivations. We take the partial derivative of Q with respect to o2 and set it to 
0 to solve for the parameter estimation step update. 


K 
DES WR C NUNT UN 
120 x5 x d (xx — | 20 (3.64) 
1 K 
= o == Y [Els] - 28 si] + ED] (3.65) 
k=2 
1 K K-1 K-1 
=[Du-2 ear + Y). (3.66) 
k=2 k=1 k=1 


The parameter estimation step update for ae when x, evolves with time 
following xy = Xx-1 + Ex is 


1 K K-1 K-1 
o? = z| ee a ust. (3.67) 
K= k= k=1 


3.5 MATLAB Examples 


In this book, we also provide a set of MATLAB code examples that implement the 
EM algorithms described in each chapter. The code examples are organized into the 
folder structure shown below: 


* one_bin\ 
sim\ 


data_one_bin.mat 
filter_one_bin.m 


expm\ 


expm_data_one_bin.mat 
expm_filter_one_bin.m 


* one mpp 
simi 


data_one_mpp.mat 
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filter_one_mpp.m 
expm\ 


expm_data_one_mpp.mat 
expm_filter_one_mpp.m 


* one_bin_two_cont\ 


* one mpp one cont 


In the case of each state-space model, the corresponding *.m" file with the code 
is self-contained and no additional path variables have to be set up in MATLAB. 
The code is written in such a manner that the “.m” file can be run directly (it loads 
the necessary data from the corresponding “.mat” file). The code in the “sim\’ and 
"expmV" folders correspond to examples running on simulated and experimental 
data, respectively. 

Estimating an unobserved state x, from a single binary observation nx gives rise 
to the simplest state-space model and EM algorithm equations. The state-space 
model with only n, was originally developed in [4]. The code for running the 
examples for this model are in the “one_bin\sim” and “one_bintexpm” folders. The 
"one binWim" folder contains the "filter one bin.m" and the *data one. bin.mat" 
files. The “.m” file contains the code and the “mat” file contains the data. We will 
use a similar naming style for all the code examples accompanying this book. 

The state-space model we considered in this chapter contained the term fo in px. 
However, we did not yet explain how it was calculated. In several studies involving 
behavioral learning experiments (e.g., [4]), Bo was determined empirically instead 
of being estimated as a separate term at the parameter estimation step. Now 


1 


Pk = 1 + e7 (odo) 


= log (2) = fo + xk, (3.68) 
1 — px 


and if we assume that xg ^ 0 at the very beginning, we have 


Po ~ 1o ( po ) (3.69) 


We can use this to calculate £o [4]. But what is po? In a typical learning experiment 
involving correct/incorrect responses, po can be taken to be the probability of getting 
an answer correct prior to any learning taking place. For instance, if there are only 
two possible answers in each trial, then po = 0.5. If there are four possible answers 
from which to choose, po = 0.25. Similarly, in experiments involving the estimation 
of sympathetic arousal from skin conductance, po can be taken to be the person’s 
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baseline probability of neural impulse occurrence. If the experiment involves both 
relaxation and stress periods, this baseline can be approximated by the average 
probability of an impulse occurring in the whole data. 

Let us first consider a basic outline of the code itself. The code takes the binary 
inputs ng for which we use the variable n. Only a few parameters need to be set in 
this particular code. One of the parameters is the baseline probability po for which 
we use the variable base prob. In general, we will set base prob to the average 
probability of ną = 1 occurring in the data. Recall that in the EM algorithm, we 
repeat the state estimation step and the parameter estimation step until the model 
parameters converge. In this code example, we use the variable tol to determine 
the tolerance level. Here we have set it to 1076 (i.e., the EM algorithm continues 
to execute until there is no change in the model parameters to a precision level in 
the order of 1076). The variable ve denotes the process noise variance. We also 
use x pred, x updt, and x_smth to denote Xkjk—1, xxjk, and xp¡k , respectively. We 
also use v pred, v_updt, and v smth to denote the corresponding variances og "m 
ok j and o, g: Prior to performing all the computations, the model parameters need 
to be initialized at some values. Here we have initialized the process noise variance 
to 0.005 and set the initial value of the x; to 0. 


base prob - sum(n) / length(n); 
tol = 1e-6; % convergence criteria 


hae) = 0,0055 
ac smeb (a)! = 0); 
b0 = log (base prob / (1 - base prob)); 


At a given iteration of the EM algorithm, the code first proceeds in the forward 


direction from k = 1,2, ..., K calculating both xx¡x-1 and xg. 

sapped) (e) =p at eE 

v_pred(k) = v_updt(k - 1) + ve(m); 

x updt(k) = get state update(x pred(k), v pred(k), bO, n(k)); 

p updt(k) = 1 / (1 + exp((-1) « (bO + x updt (k)))); 

v updt(k) = 1 / ((1 / v pred(k)) + p updt(k) « (1 - p updt(k))); 


Here the mean state update Yir is calculated using the function shown below 
(Newton-Raphson method). 


function [y] = get state update(x pred, v pred, b0, n) 


M = 50; $ maximum iterations 
ag ex vus. Tu) s 
fune = Berosil, Mi; 


df - zeros(1, M); 
sb (a) ex sm gere 
iene al ex alg (M = dl) 


um (aL) = abes) = joe = vos. (a = ooo + alte (5) )) Y 
(1 + exp(b0 + it(i)))); 
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df (i) = 1 + v pred + exp(b0 + it(i)) / ((1 + exp(bO + it(i 
DY ^ As 
aie (al s 35) = deli = Sunela) / cl (a); 
if abs(it(i + 1) - it(i)) < le-14 
sr = ale (ak <a Gl) 5 
return 
end 
end 


error ('Newton-Raphson failed to converge.'); 


end 


After proceeding in the forward direction, we reverse direction and proceed through 
k = K, (K —1),..., 1 to obtain the smoothened xz, and OK values. In the code 
shown below, the variables w and cw denote Ug and UL 444 in (3.53) and (3.54), 
respectively. 


x smth(K) = x updt(K); 
v smth(K) = v updt(K) ; 
W(K) = v smth(K) + (x smth(K) ^ 2); 
A(1:(end - 1)) = v_updt (1: (end - 1)) ./ v pred (2:end) ; 
FOBIA E le 
x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - x pred(k + 1)); 
v smth(k) = v updt(k) + (A(k) ^ 2) « (v smth(k + 1) - v pred(k 
ap 3g 
CW(k) = A(k) + v smth(k + 1) + x smth(k) « x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 


After performing state estimation at a particular iteration, we then perform param- 
eter estimation. The state estimation and the parameter estimation steps continue to 
be executed in turn until convergence. 


3.5.1 Application to Skin Conductance and Sympathetic 
Arousal 


Running both the simulated and experimental data examples produces the results 
shown in Fig. 3.3. The code running on simulated data implements the EM algorithm 
described in this chapter. The code running on experimental data, on the other hand, 
runs a slightly modified version closer to what was implemented in [25, 26] for 
estimating sympathetic arousal based on skin conductance. This version of the code 
additionally attempts to estimate the starting state xo as a separate model parameter. 

If this code is used to estimate sympathetic arousal based on skin conductance, 
the only input that is required is the sequence of ng values (denoted by the variable 
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n) that represents the presence or absence of neural impulses responsible for SCRs. 
Ideally, the sequence of neural impulses must be extracted by deconvolving the skin 
conductance data using a deconvolution procedure such as described in [47]. If, 
however, deconvolution of the skin conductance data is not possible, a simpler 
peak detection mechanism could be also used to provide these locations (peak 
detection was used in [25] and deconvolution was used in [26] for sympathetic 
arousal estimation). Also with the experimental data, and in several other examples 
that follow, we use the term “HAT” to denote “High Arousal Index" since many 
of our examples involve the estimation of sympathetic arousal from physiological 
data. The HAI is inspired by the “Ideal Observer Certainty” term in [4] and is an 
estimate of how much p; is above a certain baseline. The HAI can also be calculated 
based on x; exceeding an equivalent baseline since p; is related to xg. 

The right sub-figure in Fig. 3.3 provides an example of how sympathetic arousal 
varied for a particular subject engaged in an experiment involving different stressors. 
The experiment is described in [53]. The first three shaded backgrounds correspond 
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Fig. 3.3 State estimation based on observing one binary variable. The left sub-figure depicts 
estimation on simulated data, and the right sub-figure depicts the estimation of sympathetic arousal 
from skin conductance data. The sub-panels on the left, respectively, depict: (a) the binary event 
occurrences nx; (b) the probability of binary event occurrence pz (blue) and its estimate (red); (c) 
the state x, (blue) and its estimate (red); (d) the quantile-quantile (QQ) plot for the residual error of 
xk. The sub-panels on the right, respectively, depict: (a) the skin conductance signal; (b) the neural 
impulses; (c) the arousal state x, and its 95% confidence limits; (d) the probability of impulse 
occurrence and its 95% confidence limits; (e) the HAI (the regions above 90% and below 10% are 
shaded in red and green, respectively). The background colors on the right sub-figure correspond 
to the instruction period, a counting task, a color-word association task, relaxation, and watching 
a horror movie clip. From [32], used under Creative Commons CC-BY license 
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Sympathetic Arousal Estimation - Driver Stress 
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Fig. 3.4 Driver stress estimation. The sub-panels, respectively, depict: (a) the skin conductance 
signal; (b) the neural impulses; (c) the arousal state x, and its 95% confidence limits; (d) the 
probability of impulse occurrence and its 9596 confidence limits; (e) the HAI (the regions above 
9096 and below 1096 are shaded in red and green, respectively). The background colors in turn 
denote rest, city driving, toll road, highway, toll road, city driving, toll road, highway, toll road, 
city driving, and rest. From [32], used under Creative Commons CC-BY license 


to a period of instructions followed by two cognitive tasks. Arousal remains high 
during this period. Arousal drops significantly during the relaxation period that 
follows and briefly increases at the beginning of the emotional stressor (horror 
movie) after that. Figure 3.4 also provides an additional example of how arousal 
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varied in a driver stress experiment. The data come from the study described in [54]. 
In the experiment, each subject had to drive a vehicle along a set route comprising 
of city driving, toll roads, and highways. Figure 3.4 shows how sympathetic arousal 
varied during the different road conditions and the rest periods that preceded and 
followed the actual drive. 
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Chapter 4 M) 
State-Space Model with One Binary qe 
and One Continuous Observation 


In this chapter, we will consider the case where xg evolves with time following a 
slightly more complicated state equation and gives rise to both a binary observation 
nx and a continuous observation rg. Prior to looking into the equation derivations, 
however, as in the previous chapter, we will again first consider a few example 
scenarios where the need for such a model arises. 

In the previous chapter, we considered the estimation of a continuous-valued 
learning state xy based on correct/incorrect responses in a sequence of experimental 
trials. Based on a state-space model consisting of x, and the binary observations 
nx, the cognitive learning state of an animal could be estimated over time [4]. Note, 
however, that it is not just the correct/incorrect responses that contain information 
regarding the animal's learning state. How fast the animal responds also reflects 
changes in learning. For instance, as an animal gradually begins to learn to recognize 
a specific visual target, not only do the correct answers begin to occur more 
frequently, but the time taken to respond in each of the trials also starts decreasing 
(Fig. 4.1). Thus, a state-space model with both a binary observation ną and a 
continuous observation r was developed in [5] to estimate learning. This was an 
improvement over the earlier model in [4]. 

This particular state-space model is not just limited to cognitive learning. It can 
also be adapted to other applications as well. Human emotion is typically accounted 
for along two different axes known as valence and arousal [55]. Valence denotes the 
pleasant-unpleasant nature of an emotion, while arousal denotes its corresponding 
activation or excitement. Emotional arousal is closely tied to the activation of the 
sympathetic nervous system [56, 57]. Changes in arousal can occur regardless of 
the valence of the emotion (i.e., arousal can be high when the emotion is negative, 
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Correct/incorrect responses 


Reaction times 


Cognitive learning state 


Fig. 4.1 A monkey in a learning experiment with binary-valued correct/incorrect responses where 
reaction times are recorded. In a behavioral learning experiment, not only do the binary-valued 
correct/incorrect responses contain information regarding learning, but the time taken to respond in 
each trial also reflects changes in learning. The state-space model with correct/incorrect responses 
and reaction times was used in [5] to estimate a cognitive learning state in animal models 


as in the case of rage, or when it is positive, as in the case of excitement). As we 
saw in the earlier chapter, skin conductance is a sensitive index of arousal. Changes 
in emotional valence, on the other hand, often cause changes in facial expressions. 
Information regarding these facial expressions can be captured via EMG sensors 
attached to the face. The state-space model with one binary observation ng and one 
continuous observation rg was used in [27] for an emotional valence recognition 
application based on EMG signals. In [27], Yadav et al. extracted both a binary 
feature and a continuous feature based on EMG amplitudes and powers from data 
in an experiment where subjects watched a series of music videos meant to evoke 
different emotions. Based on the model, they were able to extract a continuous- 
valued emotional valence state x; over time. The same model was also used in [28] 
for detecting epileptic seizures. Here, the authors extracted a binary feature and 
a continuous feature from scalp EEG signals to detect the occurrence of epileptic 
seizures. Based on the features, a continuous-valued seizure severity state could 
be tracked over time. These examples serve to illustrate the possibility of using 
physiological state-space models for a wide variety of applications. 


4.1 Deriving the Predict Equations in the State Estimation 
Step 


Let us now consider the state-space model itself. Assume that x, varies with time as 
Xk = PXk-1 + £k, (4.1) 


where p is a constant (forgetting factor) and e ~ N(0, 02 y. As in the previous 
chapter, we will, for the time-being, ignore that this is part of state-space control 
system, and instead view the equation purely in terms of a relationship between 
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three random variables. As before, we will also consider the derivation of the mean 
and variance of xg using basic formulas. We first consider the mean. 


jx] = Eloxx-1 + ex] (4.2) 
= E[oxx-1] + E[ex] using (2.1) (4.3) 

= pE[xi-1] + E[ex] using (2.3) (4.4) 

= pE[xz-1] since Ele] = 0 (4.5) 
RLY = PXk-11k-1- (4.6) 


Next we consider the variance. 


V (xx) = V(pxk-1 + £k) (4.7) 
= V (pxx-1) + V (ex) + 2Cov(pxx-1, ex) using (2.4) (4.8) 
= V(pxx-1) + V (ex) since ez is uncorrelated with any of the x, terms 

(4.9) 
= o? V (xy .1) + V (ex) using (2.6) (4.10) 
^ Vou) = olog qa +02. (4.11) 


Now that we know the mean and variance of xx, we can use the fact that it is also 
Gaussian distributed to state that 


2 
RA ^ 


pGa|nia-i. rik-i) = TEk-3 (4.12) 


1 
—— e 
2 
y 210 [1 
When xx evolves with time following xy = pxx-1 + £x, the predict equations 


in the state estimation step are 


Xk|k—1 = PXk-1|k-1 (4.13) 


91 = 0^0, A (4.14) 


4.2 Deriving the Update Equations in the State Estimation 
Step 


In the current model, x, gives rise to a continuous-valued observation rg in addition 
to ng. We shall assume that x; is related to rg through a linear relationship. 
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rk = yo + YAXk + Uk, (4.15) 


where yy and yı are constants and vg ~ NO, o2) is sensor noise. Our sensor 
readings y, now consist of both rą and ng. What would be the best estimate of 
xy once we have observed yg? Just like in the previous chapter, we will make use of 
the result in (2.16) to derive this estimate. First, however, we need to note that 


KHK 


e 2% . (4.16) 


pix) = 
TO 


SLA 


This can be easily verified from (4.15). We are now ready to derive the best estimate 
(mean) for xy and its uncertainty. We will need to make use of p(rglxx), p(nk|xk), 
and p(xy|n1:—1, r1:4—1) to derive this estimate. Note that we now have an additional 
exponent term for ry in p(xy|n1:, r1:x). Using (2.16), we have 


pani rio 


| P(NK| Xk, niai, rii) pix, niis F1:k—1)P(Xk|N1:k—1; rii) 
P(nk, rk|Inya-i, T1:k—-1) 


(4.17) 
e p(nk| Xk) PKI Xk) pGacdnia-i. ri-1) (4.18) 
on? THR 
oc elt log(pr)+(1—ng) log(1l pk) y < 202 xe Viki (4.19) 
Taking the log on both sides, we have 
(rk — Yo — ixi? 
q = nylog(py) + (1 — ni) log(1 — px) 2 
20; 
_ 2 
Y + constant. (4.20) 


2 
2041 


The mean and variance of x, can now be derived by taking the first and second 
derivatives of q. Making use of (3.39), we have 


d 1k — yo ViXk) (Xk — Xkjk-1) 
1 onpi E i: HH =0. (4.21) 
dxy o% Okik—-1 


We will use a small trick to solve for x; in the equation above. We will add and 
subtract the term y; x; \,—1 in the term involving rz. 
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Vik — Yo — Yaxk + Yaxkk-i — VIXkIk1) _ (Xk — Xii) 


nk — pk + 2 = ed, 
U NS 
(4.22) 
yk — Yo — YıxXkk-1) Y? 20 Ok — Xkjk—1) 
nk — px + 32 52 9 — Xklk-1) = ETA 
v v a 
(4.23) 
2 
yi(rk — yo — YAXk(k—1).— 1 Yi 
nk — px -L E = (xk — Xk|k-1) P" + E 
(4.24) 
2 AN 
Oy (Nk — px) + YUT — Yo — VIXkIk=1) — CER ( vty 22) 
oF tk 1% 
(4.25) 
This yields the mean update 
oy 1 
Xk = Xkk-1 + 33 2 E Pk) + 0k — yo — viu]. 
Vii + 9v 
(4.26) 


Again, to clarify the explicit dependence of pz on x, and the fact that this is the 
estimate of x, having observed n1: and rj. (the sensor readings up to time index 
k), we shall say 


2 
Okik—1 

Xk|k = Xk|k-1 + —5—5 2 

Yi Odi + 9v 


[ovine Prk) + Vik — yo — viu]. 


(4.27) 
We next take the second derivative of q similar to (3.40). This yields 
dq yr 1 
5 = —Pe(l— p) — 5 - —45—. (4.28) 
dx; 9v  Okik-1 


Based on (2.21), the uncertainty or variance associated with the new state estimate 
X&|k, therefore, is 


dq -1 1 y2 -1 
ej = (E) E | 3; T Prk — Prik) + ed ; (4.29) 
dx; Okik-1 im 
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When x; gives rise to a binary observation ng and a continuous observation 
ry, the update equations in the state estimation step are 


2 
Ok k—1 
Xk|k = Xk|k-1 + 33 


A eio — Pkik) + y1 (rk — yo — yv | 
POL 1 + 97 


(4.30) 


1 Jm E 
Og = Ba + Pkik(l — pue) + z] : (4.31) 
bik 1 


4.3 Deriving the Parameter Estimation Step Equations 


In the previous chapter, we only needed to derive the update equation for the process 
noise variance o2 at the parameter estimation step. In the current model, we have a 
few more parameters. Thus we will need to derive the update equations for o, yo, 
yi, and a? in addition to the update for dz. 


4.3.1 Deriving the Process Noise Variance 


The derivation of the process noise variance update is very similar to the earlier case 
in the preceding chapter. In fact, the only difference from (3.62) is that we will now 
have px-1 in the log-likelihood term instead of xx-1. We shall label the required 
log-likelihood term Q1. 


iae — pXk— mu 


K 
Qi- = log Qro?) ) - y (4.32) 


k=2 


We take the partial derivative of Q; with respect to Hz and set it to O to solve for the 
parameter estimation step update. 


o — : Nr i| (xx — pa] =0 (4.33) 
902 — + 203 m i : 

1 K 

== y & d] — 20 [si] + e^ E [x2] (4.34) 


k=2 
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1 K K-1 K-1 
2 
= 7] Xu Y taa + Yu]. (4.35) 
k=2 k=1 k=1 


The parameter estimation step update for 07 when x, evolves with time 
following xy = oxy—1 + Ex is 


K-1 


= Xu -2p 23 Uri p? uj. (4.36) 


k=1 [seil 


4.3.2 Deriving the Forgetting Factor 


We will take the partial derivative of Q, in (4.32) with respect to p and set it to 0 to 
solve for its parameter estimation step update. 


K 
9001 (-1) . 
a 2, D| — 2xg-1(x% — oxe-1)] (4.37) 
K K 
=> 0=- y E[xexe—1] + p>. eee (4.38) 
k=2 k=2 
K-1 K-1 
--Y auo (4.39) 
k=1 k=1 


(4.40) 
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The parameter estimation step update for o when x; evolves with time 
following xy = PXk-1 + Ex is 


Il ex 
p- Y Uk ast] us| : (4.41) 


el k=l 
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4.3.3 Deriving the Constant Coefficient Terms 


We will next consider the model parameters that are related to rg. Recall from (3.57) 
that we need to maximize the expected value of the log of the joint probability 


pax N yik|8) = pO xix, O) p(x1:k10). (4.42) 


In the current state-space model, y, comprises both ng and rg. The probability term 
containing yo, y1, and o? is 


ro? 


MP. (4.43) 


K 
DUK IK. O -II 


Joli 
V 2x02 


Let us first take the log of this term followed by the expected value. Labeling this as 
Q2, we have 


= =Y> = 


(4.44) 


Ns 


02 = = E log ( (2x02) 
k=1 


To solve for yo and yı, we have to take the partial derivatives of Q2 with respect to 
yo and yı, set them each to 0, and solve the resulting equations. We first take the 
partial derivative with respect to yo. 


a 1 
oe goth nns] =0 4.45) 
=> 0= Y — wK — yı » E[ xx] (4.46) 
k=1 
K K 
=X ri yK -yı xk (4.47) 
= k=1 
K K 
yoK + y1 9 xx =X re (4.48) 
k=1 k=1 


This provides one equation containing the two unknowns yo and yı. We next take 
the partial derivative with respect to y1. 


K 
9 1 
E Y 2Epa0x — yo — rdl = 0 (4.49) 
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K K 
=> f= Xon zx] — vo Y [x] — vı 
k=1 


> 
ive 
IW 
m 
> 
TN 
ad 


K 
= TkXk|K — YO La -n 3 Uk (4.50) 
K l K K 
Yo uk thu b» Uk = b» Tk XK|K - (4.51) 
k=1 k=1 k=1 


This provides the second equation necessary to solve for yo and yy. 


The parameter estimation step updates for yo and y; when we observe a 
continuous variable ry = yo + y1xy + vy are 


IE x zi] | AT | (4.52) 


yı Dra Wei) Ue 3a EX K 


4.3.4 Deriving the Sensor Noise Variance 


The term Q2 in (4.44) also contains the sensor noise variance 02. 


do — yo— Ed 


2 
20; 


(4.53) 


We take its partial derivative with respect to o? and set it to 0 to solve for o2. 


K 
00 -K 1 | El 
= Di — yo — —0 4.54 
dol — 292 + 201 3 (rk — yo — YAXk) (4.54) 


K 
1 
= 0%= J E| (r - w- vix) ] 
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K K K K 
1 
= cot ei Y [xk] — 2yo 9 re — 2 Y RI) 
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K 
12277793 ds] 
k=1 
1 K K K K 
>= xl Sore + Ky) +y’ Un -2y ln — 2) X ruk 
k=1 k=1 k=1 k=1 
K 
+ Zu Yu]. (4.55) 
k=1 


The parameter estimation step update for o? when we observe a continuous 
variable rg = yo + Y1Xk + vy is 


K K K K 

1 

0? = xl Soret Ky Yd 9 Ue — 270 9 ri 2y1 Do exe K 
[sil k=1 k=1 k=1 


K 
+ 2yoyi Y xk | (4.56) 
k=l 


4.4 MATLAB Examples 


The MATLAB code examples for implementing the EM algorithm described in this 
chapter are provided in the following folders: 


* one bin one cont 
simi 


data one bin one cont.mat 
filler one bin one cont.m 


expm 


expm data one bin two  cont.mat 
expm filter one bin one cont.m 


Note that the code implements a slightly different version of what was discussed 
here in that the state equation does not contain 0. Code examples containing o and 
al, are provided in the following chapter for the case where one binary and two 
continuous observations are present in the state-space model. The current code can 
easily be modified if p is to be included. 
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The code for this particular state-space model is an extension of the earlier model. 
It takes in as input variables n and r that denote nz and rg, respectively. We use 
r0, r1, and vr for yo, yi, and de Shown below is a part of the code where fo is 
calculated and the model parameters are initialized. 


base prob = sum(n) / length(n); 


tol = le-8; % convergence criteria 
ve(1) = 0.005; 

x smth(1) = 0; 

POIS Ode 

i13) e s 

P AE = (5.09925 

bo = g(base prob / (1 - base prob)); 


Similar to the code examples in the preceding chapter, we also use x pred, 
x updt, and x smth to denote xj-j, Xxx, and xķįg, respectively. Similarly, 
v pred,v updt,andv smth are used to denote the corresponding variances ok k- 
of p and o, y. Just like in the earlier case as well, the code first progresses through 
the time indices k = 1,2,..., K at the state estimation step. 


x pred(k) = x updt(k - 1); 


v pred(k) = v updt(k - 1) + ve(m); 
x updt(k) - get posterior mode(x pred(k), v pred(k), r(k), rO(m), 
(m, vr(m), bO, n(k)); 


p updt(k) = 1 / (1 + exp((-1) * (bO + x updt (k)))); 
y were) e d / (CL / y pco) (Ela ^ 2) / meta) s» i sche 
(hs) a (UL = jo mjor (39) ) )) 


The update for xp¡x is calculated using the function shown below based on both 
ng and rj. 


function [y] = get posterior mode(x pred, v pred, z, r0, rl, ve, 
DO, mj 
M = 100; % maximum iterations 
aig e esos, ME 


M 
f = zeros(1, M); 
df - zeros(1, M); 
she (1) e xs gere 


for i = 1:(M - 1) 


(o spese (Wed ^ 2) o vaa se NA) B 
ie (SL) = Sabie (al) = jped= C (Gel (a = x9 = a e e E) 
+ vr « (n- (1 / (1 + exp((-1) + (b0 + it(i))))))); 
f(i) = 1+ C x vr x exp(b0 + it(i)) / ((1 + exp(bO + it( 
Me ^ aha 
tela s 1) — m ela) = (6) ^ cheval 


Tf abs (GE Ty = ate (aj) < lezl 
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yr e sbe(oL <b 30) 
return 
end 
end 


error('Newton-Raphson failed to converge.'); 


end 


The smoothing step also remains the same (there would have been a change if o 
was included). 


x smth(K) = x updt(K); 
v Smth(K) = v updt(K) ; 
W(K) = v smth(K) + (x smth(K) ^ 2); 
A(1:(end - 1)) = v_updt (1: (end - 1)) ./ v pred (2:end) ; 
FOBIA (oak) gal 
zc smt h(a) = scupd tiie) Ee AY) Gcisme hike) pred reps 
v_smth(k) = v updt(k) + (A(k) ^ 2) x (v smth(k + 1) - v pred(k 
+ 3g 
CW(k) = A(k) + v smth(k + 1) + x smth(k) « x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 


The updates for a, yo. Y1, and a? are calculated at the parameter estimation step. 


4.4.1 Application to EMG and Emotional Valence 


Running the simulated and experimental data code examples produces the results 
shown in Fig. 4.2. The experimental data example relates to emotional valence and 
EMG. Emotion can be accounted for along two different orthogonal axes known 
as valence and arousal [55]. Valence refers to the pleasant-unpleasant nature of an 
emotion. In [27], this state-space model with one binary and one continuous feature 
was used to estimate emotional valence using EMG signal features. The binary and 
continuous features were extracted based on the amplitudes and powers of the EMG 
signal. The data were collected as a part of the study described in [58] where subjects 
were shown a series of music videos to elicit different emotional responses. 
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State Estimation with Simulated Data State Estimation with Experimental Data 
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Fig. 4.2 State estimation based on observing one binary and one continuous variable. The left 
sub-figure depicts estimation on simulated data, and the right sub-figure depicts the estimation of 
emotional valence from EMG data. The sub-panels on the left, respectively, depict: (a) the binary 
event occurrences ng; (b) the continuous variable rg (blue) and its estimate (red); (c) the probability 
of binary event occurrence py (blue) and its estimate (red); (d) the state x, (blue) and its estimate 
(red); (e) the QQ plot for the residual error of xx. The sub-panels on the right, respectively, depict: 
(a) the raw EMG signal; (b) the binary EMG feature ng; (c) the continuous EMG feature rg (blue) 
and its estimate (red); (d) the probability of binary event occurrence; (e) the emotional valence 
state xx. The shaded background colors on the right sub-figure correspond to music videos where 
subject-provided emotional valence ratings were high 
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Chapter 5 Y 
State-Space Model with One Binary qe 
and Two Continuous Observations 


In this chapter, we will consider a more complicated form of the state equation—one 
that contains both a forgetting factor and an external input. We will also extend the 
earlier state-space model to the case where we now have one binary observation and 
two continuous observations. As before, however, we will first consider a scenario 
motivating the need for such a state-space model. 

Recall the example two chapters ago concerning the estimation of sympathetic 
arousal from skin conductance features. In reality, it is not just the rate of occurrence 
of neural impulses to the sweat glands that reflects changes in arousal. Other features 
in a skin conductance signal also contain arousal information. A skin conductance 
signal comprises a fast-varying phasic component superimposed on top of a slower- 
varying tonic component. The phasic component consists of all the SCRs. The 
amplitudes of these SCRs (or equivalently, the amplitudes of the neural impulses 
that generated them), in addition to their occurrence, also reflect changes in arousal 
[59]. In particular, larger SCRs reflect greater sympathetic arousal. Additionally, 
the tonic level also contains information regarding general arousal [60]. Thus, there 
are three primary sources of information in a skin conductance signal that capture 
arousal levels: (i) the occurrence of SCRs (or equivalently the occurrence of the 
neural impulses that generated the SCRs); (ii) the amplitudes of the SCRs (or the 
amplitudes of the neural impulses); (iii) the tonic component. These three make 
up one binary feature and two amplitude (continuous) features. A state-space model 
based on these three features was developed in [29], for estimating arousal from skin 
conductance. Here, a transformed version of the SCR amplitudes was interpolated 
over to derive the first continuous variable, and the tonic component was considered 
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the second continuous variable. Different algorithms are available for separating out 
the tonic and phasic components in a skin conductance signal (e.g., [61, 62]). 

A further note is also worth mentioning here. Recall the uz term in our discussion 
of state-space models at the beginning. Thus far, we have not yet considered a model 
where an external input also drives the state xz. In reality, external circumstances and 
environmental inputs all affect the way we feel. The model in [29] included such 
an external input term /;. The model was evaluated on two experimental datasets, 
one of which involved a Pavlovian fear conditioning experiment. In typical fear 
conditioning experiments, a neutral cue is paired with an unpleasant stimulus such 
as a painful electric shock. Through repeated pairing, the neutral cue alone begins 
to elicit a physiological response that is typically seen for the unpleasant stimulus 
[63]. In fear conditioning experiments, the unpleasant stimulus could also take other 
forms such a blast of air to the throat, an aversive image, or a loud sound [64, 65]. 
In [29], the neutral cues along with the unpleasant shocks were modeled as binary- 
valued indicator inputs J; that drove the sympathetic arousal state xx. 


5.1 Deriving the Predict Equations in the State Estimation 
Step 


Let us now turn our attention to the state-space model itself and assume that x, 
evolves with time as 


Xk = pxy—| + og + Ex, (5.1) 


where o is a constant and J; is an external input. The other terms have their usual 
meanings. Let us again consider how we may derive the mean and variance using 
basic statistical formulas. Since we know what the external input is, we do not treat 
it as a random variable but rather as a constant term. We first consider the mean. 


[xx] = Eloxx*-1 + ok + ex] (5.2) 
= Eloxi-1]+Elo1;] + Eleg] using (2.1) (5.3) 

= Eloxi-1] + ak + Efex] using (2.2) (5.4) 

= pE[xk-1] + al, + Elex] using (2.3) (5.5) 

= PE[xx-1] + alk since Ele] = 0 (5.6) 
ED] = exk-ijk-i +15. (5.7) 


We next consider the variance. 


V (xk) = V(oxk-i + ad + ex) (5.8) 


= V(pxx-1 d- ay) + V (ex) + 2Cov(pxx-1 + ag, ex) using (2.4) 
(5.9) 
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= V (0Xx-1 talk) + V (ex) 


since ez is uncorrelated with any of the x, or J, terms (5.10) 

= V (pxx-1) + V (ex) using (2.5) (5.11) 

= p? V (xr-1) + V(ex) using (2.6) (5.12) 

^ VG) = 0 0% ay, T 02. (5.13) 


When x; evolves with time following x, = pxx-1 + o; + €x, the predict 
equations in the state estimation step are 


Xkjk-1 = OXk-11k-1 + oi (5.14) 


Oge 1 = PO + Te: (5.15) 


5.2 Deriving the Update Equations in the State Estimation 
Step 


In this state-space model, we include a second continuous variable sz. Similar to rg, 
we will assume that sz too is linearly related to x, as 


Sk = ôo + Ô1Xk + Wk, (5.16) 


where $9 and 6; are constants and wg ~ N (0, 02) is sensor noise. Similar to the 
case of rj in (4.16), we also have 


1 — (sy 8981 x4)? 


e w (5.17) 


P(SK|XK) = 
TO. 


The procedure to derive the update equations in the state estimation step is now 
similar to what we have seen earlier. With s; included, we have yet another exponent 
term in p(xxly1:x). Therefore, 


ply) X PNK Xk) psa) pGlxk) px ÍM1:k1) rii» S1:k—1) (5.18) 
Oro 10? (sg 898114)? 
ac elogio Un) log L pG) y o 20? ird 22 
He? 


xe Pé, (5.19) 
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Taking the log on both sides, we have 


(rk — yo VixK)? (Sk — 89 — 8111)? 
202 202 


q = nk logn) + (1 — ng) log(l — px) 


LAK Xük-a)^ 


> + constant. (5.20) 
204.1 


Taking the first derivative of q and setting it to O yield 


dq yk — yo— yix). , 81 (8K — ôo — 1x4) 
= ny — pk L 3 + 2 
dxx Oy ow 
Xp — X = 
MESES a (521) 
Okik—1 


We used a trick in the previous chapter to solve for xx. We added and subtracted 
y1Xkk-1 to the term containing rg. We will do the same here. We will also add and 
subtract $1 xj, to the term containing sz. 


(Xk — Xk|k—1) yiGk — yo — Yaxk + YAXkjk-1 — YAXK|k—1) 
2 —nk— Pk + 2 
Ok k-1 oy 
61 (sx — ôo — ó1xg + Ó1XkIk-1 — Ó1XkIk— 
į 1(sk — o — ô1 Xk 2 k|k-1 — Ó1Xk|k-1) (5.22) 
Ow 
1k — Yo — YAXkk-1). , 81 (8K — ôo — Ó1Xk|k-1) 
= ny — Pk + ue 2 UM + 3 | 
OD Ow 
2 2 
y ô 
( 5+ Js Xklk-1) (5.23) 
OD Ow 
Y1 (0% — yo — VixXkk-1) , 15k — o — 81Xkj-1) 
= ny — Pet 5 EM > | 
OD Ow 
2.2 524,2 
yroz + bro 
( oe Jes Xije-1)- (5.24) 
VW 
Therefore, 
(xk — Xk\k—1) vio? +80? 
3 | + A] Or — ee) 
Okik-1 Tw 


yi Gk — yo — ViXk|k—-1) s ó1(sk — ào — 81Xk]k-1) 


2 2 
Oy ow 


= nk — Pet (5.25) 
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0202 + 8a 1 10 + 8202) 
(Xk — Xk|k-1) 5 dcos 
Okik-19 09 w 
2,2 2 
OO VOW (Tk — yo — ViXk|k-1) 
= n + 
o202 (nk — px) zy 
6102 (sy — 80 — Ô1Xkjk-1 
ue da uc (5.26) 
Oy Ow 
This yields the state update 
2 
OR k-1 


Xk|k = Xk|k—1 + 
2:22 2 22 2:55 
050, t FRM OD + 810) 


x [otoz 0n — prik) + yao (rk — yo — Y1 Xk|k—1) 
+ 8102 (sy — ôo — Biyu (5.27) 


Likewise, the second derivative yields 


d?q yz. 1 
;—-pü-po--2;-—--— (5.28) 
dx; o; OS Fkk 
And therefore, 
2 24-1 
E - Yam 
okk =| -> + Pak - Pao + T5] - (5.29) 
Okik—1 Oy Cy 


When xx gives rise to a binary observation ng and two continuous observa- 
tions ry and sx, the update equations in the state estimation step are 


2 
Okjk-1 
JEJ 2 27) 23 
ojo, + ch Vi ge + Brac) 


Xk|k = Xk]k=1 + 


x [oo — pu) + yio2 (rk — yo — YAXk|k-1) 


+ 102 (sk — 89 — Six n (5.30) 
2 1 TA rene 
wa c I DEC o s (5.31) 
Okjk—1 Oy ù 
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5.3 Deriving the Parameter Estimation Step Equations 


In this state-space model, we have the parameters o, p, Yo, Y1, 50, 51, a and o2 to 
determine. We have already seen how yo, y1, and o2 were derived in the previous 
chapter when we had rg. We will not repeat those derivations here again. Instead, we 
will only consider the derivations related to the new model parameters or changes to 
the way that earlier model parameters were derived. We will use this same approach 
of not re-deriving previous equations in the chapters that follow as well. 


5.3.1 Deriving the Terms in the State Equation 


We now have both p and « in the state equation. To determine them at the parameter 
estimation step, we will take the partial derivatives of the log-likelihood term 
containing p and a. In this case, the term we are interested in is 


K 
1 d 2 
01 = 5:5 DE 64 p cen]. (5.32) 
k=1 
Again, we set xọ = xı to permit some bias at the beginning and ignore the 


relationship through p for this boundary condition. Therefore, 


K 


01 = zl » l (a — px - en? ] + fan]. (5.33) 


We will now take the partial derivatives of Qı with respect to o and p and set them 
to 0. Let us first begin with o. 


ay T — Qe (x x amo] +2017] =0 
Ags 202 Ly k(Xk — PXk-1 k lf = 
(5.34) 
K K K 
= 0=- Y Elx] +0) nE[x alto 22 
k=2 k=2 k=1 
K K K 
= — Y ouk + oo hoa-uk +a R (5.35) 
=2 k=2 k=1 


K K K 
= PY le Ta 5 12 = M ouk. (5.36) 
k=2 k=1 k=2 
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We will next take the partial derivative of Qı with respect to p. 


al e 
= je 2 El — 2x1 (xk — pxk-1 — e1)] = 0 (5.37) 
K K K 
=> 0=—) Efx] +0) Efka] +0) nua] 
k=2 k=2 k=2 
K-1 K-1 K 
= Uk ki + p P$ Ur +a > exe 1K (5.38) 
k=1 k=1 k=2 
-1 K Kk- 
= p X UL +a X TXk-1|K = p» Uk k44. (5.39) 
= =2 k=1 


We now have two equations with which to solve for o and p. 


The parameter estimation step updates for o and œ when x; evolves with time 
following xx = pXx-1 + aly + ex are 


= =I! aa 
A =| iat Uk See] [Sg ee 


o XE a Xal EÉ, AE 


| : (5.40) 


5.3.2 Deriving the Process Noise Variance 


We next consider the parameter estimation step update for the process noise variance 
o2. The log-likelihood term containing o2 is 


Lj [os — pXk-1 — ey | 


-K 
Q2 = E log (Qro) X 


2 
E 20; 
K E — pxp_-) — e I? z| (a)? 
"m E PXk-1 al] | 1 
= — log (210? 5.41 
S og (2102) 2 202 202 (5.41) 


We take the partial derivative of Q2 with respect to o2 and set it to 0 to solve for the 
parameter estimation step update. 
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K 2 
ie M dE iue 
xa i » | (x pri cal | = 0 (5.42) 
1 K 
c2 = X » [Elx] — 2pE [xxii] + o^ E[x? 4] — 2a 1, E| xx | 
k=2 


2 K 
a 
+ 2ap I ET: H + K 2,0 
TE: K-1 K-1 K 
== | S Uk — 2p y Uk L d p? X UL — 20 toux 
k=2 k=1 k=1 k=2 


K K 
+ 200 Y Iyxi-ik «ey. (5.43) 
k=2 k=1 


The parameter estimation step update for of when x, evolves with time 
following xx = oxy—1 talk + €x is 


1 K K-1 K-1 K 
c2 = zl DU. — 20 25 Uk k+1 ps po Ux — 2a Y Iyxuk 
L K= k=1 k=2 


K K 
+ 200 Y Iyxi-ilk «eg. (5.44) 
k=2 k=1 


5.3.3 Deriving the Constant Coefficient Terms and the Sensor 
Noise Variance 


The procedure for deriving the constant coefficients 59 and ô; related to sz is similar 
to what we have seen earlier for yo and yı. The derivation of the sensor noise 


variance o is also similar to that for o7. 


The parameter estimation step updates for do, $1, and Ge when we observe a 
second continuous variable s; = ĉo + 61x, + wy are 


(continued) 
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+ 28981 You]. (5.46) 
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5.4 MATLAB Examples 


The code examples implementing the EM algorithm for the current state-space 
model are provided in the “one_bin_two_contisim” and "one bin two cont'expm" 
folders. These two directories contain the following files: 


* one bin two contV 
sim\ 


data_one_bin_two_cont.mat 
filter_one_bin_two_cont.m 
data_one_bin_two_cont_no_extern_stim.mat 
filter_one_bin_two_cont_no_extern_stim.m 


expm\ 


expm_data_one_bin_two_cont.mat 
expm_filter_one_bin_two_cont.m 
expm_data_one_bin_two_cont_no_extern_stim.mat 
expm filter one bin two cont no extern stim.m 


For both simulated and experimental data, we have provided examples with and 
without af; (the external input). Results from running the code on a simulated 
example with o7; included and on an experimental data example without o7; are 
shown in Fig. 5.1. For simulated and experimental data containing a Ig, the “.m” files 
are named "filter one bin two cont.m" and "expm filter one bin two cont.m;" 
respectively. The corresponding examples without aI, have the "no extern stim" 
suffix added to them. 

In this case, the model takes in as inputs the variables n, r, and s that denote 
Nk, Te, and sz, respectively, for estimating xg. Since there are three different 
observations, the code also has more parameters to initialize. In the code, the 
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variables ro and r1 are used for yo and y1, respectively, and so and s1 are used for 
ôo and 51. The variables vr and vs denote the corresponding sensor noise variances 
0? and o2. Finally, ve, rho, and alpha denote the process noise variance o, the 
forgetting factor p, and the o term related to Ig, respectively. Shown below is a brief 
code snippet showing the parameter initialization. 


Also, base prob (po) is still calculated based on the average probability of ng = 
1 occurring in the data. The other variables x pred, x_updt, and x smth for xj\x_1, 
Xjyjk, and xyjy remain the same, as well as the corresponding v pred, v_updt and 
v smth variables for variance. There is a sequential progression in the code through 
k = 1,2,..., K andthenthrough k = K, (K—1), ..., l at the state estimation step. 
The terms ro, r1, s0, s1, vr, vs, ve, rho, and alpha are calculated at the parameter 
estimation step. Shown below is a code snippet in the forward progression. 


x pred(k) = rho(m) * x updt(k - 1) + alpha(m) * I(k); 


v pred(k) = (rho(m) ^ 2) « v updt(k - 1) + ve(m); 

GM = jaca) / (vr(m) « vs(m) + v pred(k) * ((r1(m) ^ 99 c VE 
(m) + (s1(m) ^ô 2) x vr(m))); 

x updt(k) = get posterior mode(x pred(k), C(k), r(k), r0(m), r1(m 
Vo de(9 (i) , do), wali), ra). MS) XO (un) Ed), VE (MY) 

piupdt (ie) e / (1 + exp((-1) + (b0(m) + b1(m) x x updt(k)))); 

v updt(k) = 1 / ((1 / v pred(k)) + ((xi(m) ^ 2) / vr(m) + ((si(m 
)^2) / vs(m) + (b1(m) ^ 2) « p updt(k) « (1 - p updt (k)) 


The code where we proceed in the reverse direction at the state estimation step is 
shown below. While it is largely similar to what we saw in an earlier chapter, now 
the variable rho is also included. 


x smth(K) - x updt(K); 
v smth(K) = v updt(K) ; 
W(K) = v smth(K) + (x smth(K) ^ 2); 
A(1:(end - 1)) = rho(m) « v updt(1:(end - 1)) ./ v pred(2:end) ; 
for k= IK = d)vi51)sl 
x smth(k) = x updt(k) + A(k) « (x smth(k + 1) - x pred(k + 1)); 
v smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - v pred(k 
ar 35) yg 
CW(k) = A(k) x v smth(k + 1) + x smth(k) « x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
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Note that in the examples where an external input is absent, alpha is not calculated. 
The state estimation step and the parameter estimation step are performed in turn 
until convergence. 


5.4.1 Application to Skin Conductance and Sympathetic 
Arousal 


This state-space model with one binary and two continuous observations was used 
in [29] for estimating sympathetic arousal from skin conductance. In the model, 
the tonic component made up the continuous variable s. The other continuous 
variable rg was derived somewhat differently. SCR amplitudes can have a skewed 
distribution which a log transformation can help correct. Therefore, the log of the 
SCR amplitudes was taken and interpolated over to generate rg. 

A further point is to be noted with experimental data. The estimated state x; 
can occasionally overfit to one of the continuous variables [29]. Consequently, an 
additional constraint was applied to allow the parameters corresponding to rg and sz 
(i.e., yo, y1. E ôo, 91, and 02) to update only if the sensor noise variance estimates 
did not differ by more than a certain amount. Details of this can be found in [29]. 
This constraint prevented one of the sensor noise variance estimates from being 
driven down at the expense of the other (which takes place during overfitting). 

If the external inputs are unknown, the version of the code without o7; can 
be used. The experimental results in Fig. 5.1 are from a case where o; is not 
considered. The data come from the stress experiment in [53] which we also 
considered two chapters ago. The portion of the experiment considered here consists 
of the cognitive stressors, relaxation, and the horror movie clip. The state estimates 
are high during the cognitive stressors and thereafter gradually diminish. However, 
the increase seen in the HAI at the beginning of the horror movie clip is quite 
significant. 

Data from the Pavlovian fear conditioning experiment in [66] are taken for 
the experimental code example containing the o7, term. The results are shown 
in Fig. 5.2. The experiment is described in detail in [67, 68]. In a typical fear 
conditioning experiment, a neutral cue is paired with an unpleasant stimulus such 
as a painful electric shock. Through repeated pairing, a subject begins to display 
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State Estimation with Simulated Data State Estimation with Experimental Data 
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Fig. 5.1 State estimation based on observing one binary and two continuous variables. The left 
sub-figure depicts estimation on simulated data, and the right sub-figure depicts the estimation of 
sympathetic arousal from skin conductance data. The sub-panels on the left, respectively, depict: 
(a) the probability of binary event occurrence p; (blue) and its estimate (red) (the green and black 
dots above at the top denote the presence or absence of binary events, respectively); (b) the first 
continuous variable rg (blue) and its estimate (red); (c) the second continuous variable s; (blue) 
and its estimate (red); (d) the state x; (blue) and its estimate (red) (the cyan and black dots denote 
the presence or absence of external binary inputs, respectively); (e) the QQ plot for the residual 
error of xy. The sub-panels on the right, respectively, depict: (a) the skin conductance signal (the 
green and black dots on top denote the presence or absence of SCRs, respectively); (b) the phasic- 
derived variable rg (green solid) and its estimate (dotted); (c) the tonic level sg (pink solid) and 
its estimate (dotted); (d) the arousal state xy and its 95% confidence limits; (e) the probability of 
SCR occurrence p; and its 95% confidence limits; (f) the HAI (the regions above 90% and below 
10% are shaded in red and green, respectively). The background colors on the right sub-figure 
correspond to the instruction period, a counting task, a color-word association task, relaxation, 
and watching a horror movie clip. From [32], used under Creative Commons CC-BY license 


a response to the neutral cue alone. In the experiment in [66], two types of cues 
were used. One of the cues never preceded the electric shock. This is labeled the 
CS- cue. The second cue, labeled as CS+, preceded the shock 50% of the time. The 
code example sets J; = 1 at the locations of the neutral cues and the shocks. Other 
types of inputs may also be considered for J;. Figure 5.2 depicts the averages for 
the CS- trials, the CS+ trials that did not contain the electric shock, and the CS 
trials that did contain the shock. As seen in Fig. 5.2, for this particular subject, the 
CS+ with the shock elicited the highest skin conductance and sympathetic arousal 
responses. The CS- trials had the lowest skin conductance and arousal levels, and 
the CS+ without the shock had responses in-between these two. 
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Sympathetic Arousal Estimation - Fear Conditioning 
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Fig. 5.2 State estimation in Pavlovian fear conditioning. The sub-panels, respectively, depict: (a) 
the skin conductance signal zz; (b) the phasic-derived variable (green solid) and its estimate 
(dashed); (e) the tonic level są (mauve solid) and its estimate (dashed); (d) the probability of 
SCR occurrence p, (the cyan and black dots on top denote the presence or absence of SCRs, 
respectively); (e) the arousal state x; (the green and black dots denote the presence or absence 
of external binary inputs, respectively); (f) the averages corresponding to different trials for skin 
conductance (CS— —green, CS+ without the shock—mauve and CS+ with the shock—red); (g) 
the same averages for the arousal state x;. © 2020 IEEE. Reprinted, with permission, from [29] 
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Chapter 6 N 
State-Space Model with One Binary, Two — « 
Continuous, and a Spiking-Type 

Observation 


Spiking-type observations are occasionally recorded in experiments. For instance, 
neural spiking activity may be recorded from a macaque monkey engaged in a 
learning experiment or an EKG signal may be recorded from a human subject in 
an experiment. In such instances, we can model the spiking-type variable using 
a conditional intensity function (CIF). The CIF is similar to the rate parameter 
in a Poisson distribution but is more general. With spiking-type observations, we 
usually assume that our state variable x; affects the rate of spiking through the CIF. 
Now we need to estimate x, at each time index k. In the case of a spiking-type 
variable, we typically observe the spiking over a short interval corresponding to 
time index k. For instance, in the case of a macaque monkey performing a behavioral 
learning task, we may observe neural spiking over a period of several hundred 
milliseconds corresponding to each trial k. Each trial duration is then divided into 
smaller bins indexed over j. Since the spiking-type variables are binary, we assign 
either my; = 0 or my,; = 1 within the interval k for each of the smaller time 
bins j based on spike occurrence. Shown below is an example CIF Az, ; used in an 
experiment where a monkey's learning state was estimated from measurements that 
included neural spiking [6]. 


Mj = Oot wrt Ya DK jos (6.1) 
In general, the specific form of the CIF depends on the type of application. In this 


chapter, we will derive the state and parameter estimation step equations for a model 
where a spiking-type variable characterized by a general CIF Az, j is observed along 
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with one binary and two continuous variables. We will, however, first consider the 
need for such a state-space model. 

In the preceding chapter, we looked at a state-space model for estimating 
sympathetic arousal based on one binary and two continuous skin conductance 
observations. The occurrence of SCRs made up the binary observation nz. The 
continuous observations comprised of a transformed version of the SCR peaks 
and the tonic level. In reality, the sympathetic nervous system affects a number 
of organs, not just the skin. We can go to any one of these organs to extract 
features to estimate arousal. However, not all these organs (or the corresponding 
physiological signals) are conveniently accessible. The heart is one organ affected 
by sympathetic activation for which the corresponding signals can be measured 
easily (e.g., using an EKG). Now sympathetic drive is known to increase heart rate 
and the force of ventricular contraction [69]. The heart, however, is innervated by 
both sympathetic and parasympathetic fibers and also has its own pacing mechanism 
[70]. Consequently, a precise extraction of the sympathetic activation component 
from an EKG signal is a challenge. In [31], a state-space model based on three 
skin conductance features (the features just referred to) and EKG signals modeled 
as spiking observations was used to estimate sympathetic arousal. Here, the model 
assumed that increased sympathetic arousal caused EKG inter-beat intervals (known 
as RR-intervals) to decrease (i.e., caused heart rate to increase). The CIF was based 
on the history-dependent inverse Gaussian (HDIG) probability density function for 
RR-intervals [71, 72]. The state-space model could be used for wearable healthcare 
applications (Fig. 6.1). Post-traumatic stress disorder (PTSD), for instance, is known 
to involve symptoms of Ayperarousal [73], while major depression is known 
to involve low levels of arousal [74]. Thus, a wearable device based on skin 
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Fig. 6.1 A wearable sensing system for decoding sympathetic arousal. The sweat glands are 
innervated by sympathetic nerve fibers, and the heart is innervated by both sympathetic and 
parasympathetic fibers. This information from skin conductance and heart rate can be used to 
estimate sympathetic arousal. From [26], used under Creative Commons CC-BY license 


6.2 Deriving the Update Equations in the State Estimation Step 69 


conductance and heart rate measurements for monitoring arousal could be used to 
help care for such patients. 

We also make another notable observation here. The phenomena occurring within 
the human body and brain are rather complex. Thus, it is likely that no single type 
of physiological signal or feature captures all the necessary information regarding 
latent physiological states. If, for instance, both emotional valence and arousal are 
to be decoded, features from a number of signals could be considered [58, 75-78]. 
Signals such as EMG [27, 79-82], heart rate [83-87], respiration [88—92], and blood 
flow signals within the brain (functional near infrared spectroscopy) [93-97] all 
contain information regarding phenomena such as emotion and cognitive effort. 


6.1 Deriving the Predict Equations in the State Estimation 
Step 


We have already considered three different cases for the state equation: (1) the simple 
random walk; (ii) the random walk with a forgetting factor o; (iii) the random walk 
with a forgetting factor p and an external input J;. You would have noticed by now 
that changes to the state equation primarily affect the predict equations in the state 
estimation step and not the update equations. The three cases we have considered 
thus far cover most of the applications that are encountered in typical physiological 
state estimation problems. In the current state-space model, we will assume that 
xy evolves with time following one of the state equations we have already seen. 
Thus no new predict step equations have to be derived. These signals could be used 
for wearable healthcare applications. A study of how different external stimuli also 
affect emotion could lead to novel neuromarketing strategies as well [98]. 


6.2 Deriving the Update Equations in the State Estimation 
Step 


When dealing with a spiking-type observation, we first split our observation interval 
at time index k into smaller segments and index these smaller bins as j = 
1, 2, ..., J. The joint probability of the spikes over the J observation bins is then 
[99] 


J . EE : 
plz 1, MED, .. K lx) = CX I=! EAI AIM ¡4 GA (6.2) 


Recall from (5.18) that when we had one binary and two continuous observations, 
the posterior density was 


Pl yx) e PNK XK) PK XK) pG|xk) PORN K-15 riis S1:k1)- (6.3) 
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Now that we have the spiking-type observation, we will include p(mx,1, mx,2, .. 
my, j|xy) in p(xxly1:x) as well. Therefore, 


D 


ero? — (589311192 
pOulyia) oc eM BP r7 nolog(l7PO xe 205 xe 2% 
= (x 3g? 
J . e —M——— 
x ein lea sme AGA yg EG (6.4) 


The procedure for deriving the update equations is again similar to what we have 
seen thus far. As before, we will take the first derivative of the exponent term, set 
it to O, and solve for x; to obtain the mean. We will then take the second derivative 
to obtain the uncertainty or variance associated with the estimate. Taking the log of 
the posterior density and setting the first partial derivative to 0 yield 


dq — —(xk — Xkik-1) zu (rk — yo — YAxk) 


+ (nk — Pk) 
J 
$ M > 1 dL: 
n 1 Gk D 1Xk) i kj (my, j — Àk, jA) = 0. (6.5) 
07 jaa ^j AX 


Solving for xy is now similar to what we saw in the earlier chapter. We simply 
need to add and subtract yı xx\4—1 and 81 xx¡-1 from the terms containing ry and sz, 
respectively. The second partial derivative is 
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j 2 d 2 ; d A Xo iki Ak, jA) 
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Thus the updates for xy; and og , turn out to be 
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Note that the equations may simplify further depending on the specific form of the 
CIF. Here we have provided the derivations for the general case. 


When xx gives rise to a binary observation ng, two continuous observations 
ry and sy and a spiking-type observation m, j characterized by the CIF Ax j, 
the update equations in the state estimation step are 
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6.3 Deriving the Parameter Estimation Step Equations 


The state-space model we consider here is an extension of what we considered in 
the previous chapter that contained one binary and two continuous observations. 
Therefore, the only new parameter estimation step equations we need to derive are 
for the spiking-type variable. 
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6.3.1 Deriving the Coefficients Within a CIF 


A CIF can take different forms depending on the type of application. For instance, 
when neural spiking data are involved, log(A,, j) may be expressed as a linear sum 
of history-dependent terms and x, as in (6.1). If this is the case, we would have 
to determine y and the @,’s at the parameter estimation step. When heartbeats 
are modeled as a spiking-type variable, the CIF involves an inverse Gaussian 
distribution and could be related to x, through its mean [31]. Thus, the terms to 
be derived at the parameter estimation step when a spiking-type variable is present 
are application-specific. In general, due to the rather complicated nature of a CIF, 
the parameter estimation step updates do not have neat closed-form expressions. 
Instead, the parameters have to be chosen to maximize the expected log-likelihood 


K 
0= Y El togQu. j Adm, - UH (6.11) 
k=1 j=1 
The form of Q can be deduced from (6.2). The trick to maximizing Q is to perform 
a Taylor expansion around the mean xy, = Elx] for each of the summed terms. 
Therefore, when the expected value is finally calculated, we will end up with terms 
like Elxx — xxx] and El (xx — Yuk 17) in the expansion. Now 


LYC — XK l = Elxx] — xx¡k based on (2.2) (6.12) 


= Xk|K — Xk|K = 0, (6.13) 


and JE[ (xx — xx x] is the variance og x: These two facts will greatly help simplify 
the calculation of Q. 

Let us now perform the Taylor expansion around xj, [6]. The summed term 
within the expected value simplifies to 
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Taking the expected value, we have 
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Note the terms E[xy — xxx] and Ef (xk — Yuk 17) in the expression above. The first 
of these is 0, and the second is the variance o x: Therefore, 


¿[ log(ax,j Amg, j — Ax jik A] © log ji Amk, j — Ax jk ^ +0 


Consequently, Q approximately simplifies to 
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In general, Q will have to be maximized with respect to the model parameters in the 
CIF using numerical methods. 


The parameter estimation step updates for the terms in a CIF Az, ; when we 
observe a spiking-type variable m, j are chosen to maximize 
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(continued) 
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1 1 8214 jlK my. j (OAK K S 2 
+ > (mx, j — Ask A) — = = DK: 
bem ax? s 7 MK OXK de 


(6.18) 


6.4 MATLAB Examples 


MATLAB code examples for simulated and experimental data for the state-space 
model with one binary, two continuous, and one spiking-type observation are 
provided in the folders shown below: 


* one bin two cont one spk 
sim\ 


data_one_bin_two_cont_one_spk.mat 
filter_one_bin_two_cont_one_spk.m 


expm\ 


expm_data_one_bin_two_cont_one_spk.mat 
expm_filter_one_bin_two_cont_one_spk.m 


6.4.1 Application to Skin Conductance, Heart Rate and 
Sympathetic Arousal 


The state-space model described in this chapter was used in [31] to estimate 
sympathetic arousal from skin conductance and heart rate measurements. The skin 
conductance observations are the same three that were used in [29] (discussed in the 
previous chapter). Thus, the only new observation added here relates to heart rate 
for which some additional discussion is necessary. 

The code examples estimate arousal from the four observations related to skin 
conductance and heart rate. The R-peaks in the EKG signals are taken to form the 
spiking observations. If L consecutive R-peaks occur at times u; within (0, T] such 
that O < uy < u2 < ... «ur < T, and hı = uj — uj_ is the JT RR-interval, the 
HDIG density function for the RR-intervals at f > uj is 


(6.19) 
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where q is the model order, 6,..; is related to the variance, and the mean is 


q 
u = bo +) Ah + Nk, (6.20) 


i=l 


where 7 is a coefficient to be determined. Accordingly, a change in sympathetic 
arousal xg causes the mean of the HDIG density function to shift (i.e., heart rate 
speeds up or slows down depending on the arousal level). The CIF Ax, j is 


8 (tx, lux, j) 
1— fie! geluk dz 


Apa (6.21) 


where ux,; is the time of occurrence of the last R-peak prior to fj, ;. The CIF 
Ax,j is calculated every A = 5 ms [23, 71]. Since skin conductance is typically 
analyzed at 4 Hz (t, = 250 ms), there are 250/5 = 50 smaller observation bins j 
for heart rate at each time index k. Due to computational complexity, the 0;'s were 
estimated separately in an offline manner using maximum likelihood. Now the work 
by Barbieri et al. [71] was one of the earliest to perform point process analysis of 
EKG RR-intervals using the HDIG density function.! The EM algorithm in [31] 
was executed for several different values of n. and the best one was selected based 
on a maximization of the log-likelihood term in (6.17). Note also that since the 
experimental code example involves skin conductance and heart rate with A = 5 
ms bins, the heart rate observations need to be provided to the code in a manner 
similar to that contained in the .mat file. 

The other aspects of the code and the variable names are similar to what 
was described in earlier chapters. Running the code examples on simulated and 
experimental data yields the results shown in Fig. 6.2. The experimental data results 
are from the Pavlovian fear conditioning experiment in [100]. As shown in the 
figure, the CS+ trials with the electric shock have the highest average responses, 
while the CS- trials have the lowest average responses for the subject considered. 
The CS+ trials without the shock have an intermediate response. 


T The code for calculating the 6;’s for a series of RR-interval measurements via maximum 
likelihood is provided at http://users.neurostat.mit.edu/barbieri/pphrv. 
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Fig. 6.2 State estimation based on observing one binary, two continuous, and one spiking-type 
variable. The left sub-figure depicts estimation on simulated data, and the right sub-figure depicts 
the estimation of sympathetic arousal from skin conductance and heart rate data. The sub-panels 
on the left, respectively, depict: (a) the probability of binary event occurrence pz (blue) and its 
estimate (red) (the green and black dots above at the top denote the presence or absence of binary 
events, respectively); (b) the first continuous variable rj (blue) and its estimate (red); (c) the second 
continuous variable s; (blue) and its estimate (red); (d) the state x, (blue) and its estimate (red) 
(the cyan and black dots denote the presence or absence of external binary inputs, respectively); 
(e) the simulated RR-interval sequence (orange) and the fit to the HDIG mean; (f) the QQ plot for 
the residual error of xg. The sub-panels on the right, respectively, depict: (a) the skin conductance 
signal zg; (b) the probability of SCR occurrence pz (the green and black dots on top denote the 
presence or absence of SCRs, respectively); (c) the phasic-derived variable (green solid) and its 
estimate (dotted); (d) the tonic level sz (pink solid) and its estimate (dotted); (e) the arousal state xg 
(the cyan and black dots denote the presence or absence of external binary inputs, respectively); (f) 
the RR-interval sequence (orange) and the fit to the HDIG mean; (g) the averages corresponding to 
different trials for skin conductance (CS— —green, CS+ without the shock—mauve and CS+ with 
the shock—red); (h) the same averages for the state. From [31], used under Creative Commons 
CC-BY license 
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Chapter 7 Y 
State-Space Model with One Marked qe 
Point Process (MPP) Observation 


Thus far we have considered binary observations and continuous observations in our 
state-space models. With binary observations, we do not consider the magnitudes of 
the binary-valued events (since each is just a O or a 1) but are merely interested in 
the event occurrences. Consequently, we can treat the spiking-type observations in 
the earlier chapter as binary-valued as well. There too, our concern was primarily 
with the occurrence of the cardiac contractions and the accompanying spikes in an 
EKG signal, but not the actual amplitudes of the spikes. But what happens when 
we observe a point process that is not just a sequence of zeros and ones but rather 
is a sequence of zeros and real-valued amplitudes? Such a point process forms a 
marked point process (MPP). These are encountered frequently in physiological 
state estimation applications as well. For instance, the sequence of neural impulses 
underlying a skin conductance signal forms an MPP (Fig.3.2). So do pulsatile 
hormone secretory events. In this chapter, we will learn how to derive the state 
and parameter estimation step equations when the state-space model contains MPP 
observations. 

In this chapter also, we will begin by considering a motivating example. Now we 
can build many models ranging from simple to complex to account for physiological 
phenomena. Any mathematical abstraction of a real-world system will have some 
imperfections to it and will not be able to fully account for all of the data. 
Occasionally, in engineering systems, we will encounter cases where a simpler 
model performs better than a more complex model. The estimation of sympathetic 
arousal from skin conductance is one such case. The state-space model with one 
binary and two continuous observations is quite complex [29]. However, despite its 
complexity, it is somewhat imperfect in that it interpolates over a log-transformed 
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version of the SCR amplitudes. A more natural way to account for phasic skin 
conductance variations is to model the underlying neural impulses as an MPP [32]. 
This eliminates the need for two continuous variables and is simpler. 

A further algorithmic detail is also worth noting here. Mathematical models 
of real-world systems will always have some limitations. The limitations may be 
in the model itself or have to do with issues that arise during computation. This 
book focuses on the estimation of unobserved physiological quantities that are 
related (fully or partially) to point process observations. Occasionally, when we 
have both binary and continuous variables involved, the EM algorithm can have 
a tendency to converge to locations where there is a near-perfect fit to one of the 
continuous variables (1.e., overfitting occurs). The state-space model with one binary 
and two continuous observations has this tendency to overfit on experimental data. 
Consequently, additional constraints have to be put in place to control it [29]. This 
issue can also occur in the model with one binary and one continuous observation. 
The use of the MPP framework circumvents the need to have a continuous variable 
and thus avoids the need for external overfitting control. Thus the simpler MPP state- 
space model for estimating arousal based on skin conductance performed quite well 
in comparison to others [32]. 


7.1 Deriving the Update Equations in the State Estimation 
Step 


In this chapter also, we will assume that x, evolves with time following one of the 
state equations we have already seen. Thus no new predict step equations have to be 
derived. 

Recall from (3.21) that the PDF of a single (Bernoulli-distributed) binary 
observation n; is 


P(nglxn) = př — p ^. (7.1) 


This same density function can be written as 


l— py ifng=0 
pda) = 4 PE (7.2) 
Dk ifn, = 1. 


In reality, we could derive our state estimation step update equations based on (7.2) 
as well. For instance, if we observed nz = 0 at time index k, the posterior density 
would be 


Oy? Oo? 
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(7.3) 
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where we have substituted ng = 0 into the exponent of the first term. We could next 
take the first and second derivatives of the exponent to obtain the corresponding 
state estimation step update equations for xj, and ok g- We could also do the same 
for ng = 1. In the case of ng = 1, we would have 


Gp oq? 10? 
p(xx ini) oc e'h log(pi) +14) log(17 BG x < toj, = LE) x e dej. . 
(7.4) 
Finally, we could express the update step equations for the two different cases based 
on an if-else condition. This would be of the form 


if ny — 0, 
Xijk es (7.5) 
One Sex (7.6) 
if ny = 1, 
Xkk = oe. (7.7) 
Ce (7.8) 


In the case of an MPP where we have non-zero amplitudes only at the instances 
where point process events occur, the density function for the observations is 


1 — py = elel—pe) ifn, =0 
PNK XK) = vor? K NE 0? 
Pk 1 - 202 = clog(px) l e 202 ifn, = 1, 


2107 y 2102 


(7.9) 


where the point process event amplitudes (i.e., the marks) rg are assumed to be 
linearly related to xy through ry = yo + yi1xy + vy, where vg ~ N (0, 0?) 1s sensor 
noise. 

Let us now proceed with calculating the update step equations for the two cases 
where n, = 0 and nz = 1. First consider n, = 0. Based on (7.9), the posterior 
density is 


: 2 
RA MD 


c2 
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(7.10) 


We can now take the log, take its derivative, and set it to 0 to solve for the mean. 
This yields 
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dqi 1 2(Xk — Xklk1) — 


= k(l — pk) 0 (7.11) 
du  ü-po P P 26], | 
(Xk — Xk|k—1) 
A (7.12) 
Okjk-1 
Xk KK) TOOL D) (7.13) 
Xy KEK) Og 0 Pr) since ng = 0. (7.14) 


Interestingly, this is the same as (3.38) where we only had one binary observation 
nx in the state-space model. Let us now calculate the variance by taking the second 
derivative. 


-1 
= 1 s 7.15 
d cha DUL — px) (7.15) 


Again, interestingly, this turns out to be the same as (3.40) where we only had one 
binary observation. Therefore, when a point process event does not occur (1.e., when 
ny = 0), our state estimation step update equations are similar to the case where we 
only had one binary observation in the state-space model. 

We will next consider the case when nz = 1. Note that we will then have the rg 
amplitude term as well. Based on (7.9), the posterior is now 
Ty? 


KHK] » 
pGklyya) x eSP) xe 27 xe hi. (1.16) 


Taking the log and proceeding to take the first derivative, we have 


dq; 1l Virk — yo — VIXK) Gk — xkgk-i) 
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Since ng = 1, we will replace (1 — p) with (ng — px). Therefore, 
dq? Vik — yo — YIXK) (Xk — Xk]k-1) 
2 = (ny — pe) + wae ae =0. (7.18) 
dx Oy DUL 1 


This is the same as (4.21) where we had both a binary variable and a continuous 
variable in the state-space model. Therefore, based on (4.26), the mean update for 
Xk is 
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7.2 Deriving the Parameter Estimation Step Equations 
Also, when we take the second derivative, we end up with 
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just like (4.28). 
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(7.20) 


This provides an interesting insight. In the case of an MPP, the state estimation 
step update equations switch between those where one binary variable was observed 
and where both a binary variable and a continuous variable were observed. This 


switching occurs depending on whether ng = 0 or nz = 1. 


When x; gives rise to MPP observations comprising of the pairs (nz, rg), the 


update equations in the state estimation step are 


if ny — 0, 

Xkjk = KHK) + Og (k — Prik) (1.21) 
1 =i 
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7.2 Deriving the Parameter Estimation Step Equations 


The only changes that occur at the parameter estimation step relate to yo, y1, and 
02. Parameter estimates for other variables such as the process noise variance o2 do 


not change. 
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7.2.1 Deriving the Constant Coefficient Terms 


Recall from (4.43) that when we observed one binary variable and one continuous 
variable, the probability term containing yo, y1, and o2 required at the parameter 
estimation step was 


K 1 ri NO? 
pila. 9) =| | e a. (7.25) 
pap Am 
Notice that the product is over k = 1,2,..., K. This is when we observed a non- 


Zero ry at each point in time. When we observe an MPP variable as modeled in (7.9), 
ry shows up only at the time indices where n, = 1. Let us assume that the point 
process events occur at time indices K C {1,2,..., K}. Therefore, in the case of 
an MPP, the probability term we are interested in at the M-step will be 
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where the product is only over the specific indices K rather than everywhere. The 
corresponding log-likelihood term is therefore 
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We can now proceed by taking the partial derivatives with respect to yo, y1, and 07, 
setting them to 0 and solving. This yields 
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Note that all three equations shown above are similar to the case where a continuous 
variable was always present. Now, however, the summations are only over K. Thus 
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the parameter estimation step updates for yo, y1, and o? are very similar to what we 
have seen before. 


The parameter estimation step updates for yo, y; and es when we observe an 
MPP variable with the amplitudes modeled as rg = yo + yj xp + vy are 


PIE IKI Zeer T f 2 ek "k | (7.31) 
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7.3 MATLAB Examples 


The MATLAB code examples for estimating xg from a set of MPP observations are 
provided in the following folders: 


* one mpp 
sim\ 


data_one_mpp.mat 
filter_one_mpp.m 


expm\ 


expm_data_one_mpp.mat 
expm_filter_one_mpp.m 


The code examples estimate x, based on the inputs n, and rg denoted by the 
variables n and r. A few differences are to be noted in this code compared to the 
previous examples. In the previous MATLAB examples, we had the predict, update, 
predict, update, etc. steps executed repeatedly for k — 1,2,..., K. However, when 
we have MPP observations, we have two different filter update equations depending 
on the value of ng. The rg amplitudes are only taken into account when nz = 1. 
Therefore, the state estimation step contains the following: 
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x updt(k) = get posterior mode(x pred(k), v pred(k), r(k), r0(m), 
ri(m), vr(m), bO, n(k)); 
p updt(k) = 1 / (1 + exp((-1) * (bO + x updt (k)))); 


ae (ale) == 
v updt(k) = 1 / ((1 / v pred(k)) + p updt(k) « (1 - p updt (k))) 
elseif (n(k) == 1 
y wicks (is) a al 7 (al / vaa) «s (eu) ^ 2) / sm) sx» 
p updt(k) « (1 - p updt(k))); 
end 


The state update, also based on an if-else depending on the value of nx, is calculated 


using the get posterior mode(...) function shown below: 
function [y] = get posterior mode(x pred, v pred, z, r0, rl, vr, 
BOr m) 
Mos 100; % maximum iterations 
y = NaN; 
it = zerosi(1, M) 


f = zeros(1, M); 
df - zeros(1, M); 


she (a) e se gre; 
for d$ - 0M = 4 
ake (in == 0) 
(= Ww joist! 
f(i) = it(i) - x pred - C « (n - exp(bO + it(i)) / (1 
+ exp(b0 + it(i)))); 
df(i) = 1+ C * exp(bO + it(i)) / (1 + exp(bO + it(i) 
)) ^ 25 
elseif im se 1) 
(Gc Wissel Y (Gel ~ 2) cx yace ve), 
Es) e atte (sl) = s jewel = € (mil a (m = x = s oc 
es gere) + wem x» fum > (Cal / (1 + exp((-1) * (bO + it(i))))))); 
df (i) = 1 + C x vr x exp(b0 + it(i)) / ((1 + exp(bO + 
icu») ^ Da 
end 
aie (aL s 3D) = del(a) = sea) / che (al) p 
she cerae aL ss 30) > ade (y) dei 
yr m abE(oL sb yy 
return 
end 
end 


error('Newton-Raphson failed to converge.'); 


end 


7.3 MATLAB Examples 85 


The other variables used in the code largely remain the same. For instance, we 
still use x pred, x updt, and x_smth to denote Xkjk—1, XKE, and xx\K,, respectively, 
and v_pred, v_updt and v_smth to denote the corresponding variances of k-p og k 


2 
and OK: 


7.3.1 Application to Skin Conductance and Sympathetic 
Arousal 


As stated earlier, the sequence of neural impulses underlying the phasic variations in 
a skin conductance signal forms an MPP. This sequence of impulses is extracted via 
deconvolution. In the code example, the input (i.e., the deconvolved neural impulse 
train) is provided through the variables n and r. The variable r(k) has a non- 
zero amplitude whenever n(k) is equal to 1. The r(k) amplitudes are not taken 
into account when n (k) is 0. Running the MATLAB examples on simulated and 
experimental data produces the results shown in Fig. 7.1. The filter was used in [32] 
for estimating sympathetic arousal from deconvolved skin conductance data. The 
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Fig. 7.1 State estimation based on observing one MPP variable. The left sub-figure depicts the 
estimation on simulated data and the right sub-figure depicts the estimation of sympathetic 
arousal from skin conductance data. The sub-panels on the left, respectively, depict (a) the MPP 
observations (blue) and the estimated rg (red), (b) the point process event occurrence probability 
Pk (blue) and its estimate (red), (c) the state xg (blue) and its estimate (red), and (d) the QQ plot for 
the residual error of xx. The sub-panels on the right, respectively, depict (a) the skin conductance 
signal, (b) the neural impulses underlying phasic variations, (c) the arousal state x; and its 95% 
confidence limits, (d) the probability of impulse occurrence pz and its 95% confidence limits, and 
(e) the HAI (the regions above 90% and below 10% are shaded in red and green, respectively). The 
background colors on the right sub-figure correspond to the instruction period, a counting task, a 
color-word association task, relaxation, and watching a horror movie clip. From [32], used under 
Creative Commons CC-BY license 
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Fig. 7.2 Driver stress estimation. The sub-panels, respectively, depict (a) the skin conductance 
signal, (b) the neural impulses, (c) the arousal state x, and its 95% confidence limits, (d) the 
probability of impulse occurrence and its 9596 confidence limits, and (e) the HAI (the regions 
above 9096 and below 1046 are shaded in red and green, respectively). The background colors in 
turn denote rest, city driving, toll road, highway, toll road, city driving, toll road, highway, toll 
road, city driving, and rest. From [32], used under Creative Commons CC-BY license 


results on experimental data shown in the figure are based on the study described 
in [53] (seen in the earlier chapters as well). The study involved different types of 
stressors interspersed by periods of relaxation. The results of using the same code 
on the driver stress data in [54] for a particular subject are shown in Fig. 7.2. 
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Chapter 8 M) 
State-Space Model with One MPP and P 
One Continuous Observation 


In this chapter, we will derive the EM algorithm equations for a state-space model 
having an MPP and a continuous-valued variable as its observations. Before looking 
at the state-space model itself and the equation derivations, we will again first 
consider a scenario for where the need for such a model arises. We stated earlier that 
the human body is comprised of multiple internal sub-systems that are networked 
with one another. The sub-systems perform specialized functions and all work in 
unison to maintain life. Now multiple functions within the body are regulated by 
the endocrine system. The endocrine system governs the secretion of a number of 
hormones that act on different target cells in the body. These hormones largely serve 
as messengers and help coordinate activities between sub-systems within the body. 
Functions that hormones are involved in include metabolism, the regulation of blood 
glucose and appetite, and playing a role in the body’s immune and stress responses, 
to name a few [101]. 

The secretory mechanism is pulsatile in the case of a number of hormones. 
Cortisol is one such example [38]. One of the major functions of cortisol is to 
raise blood glucose levels in response to external stressors [102, 103]. When the 
brain interprets sensory inputs as requiring cortisol secretion, the hypothalamus 
begins to secrete the hormone CRH (corticotropin-releasing hormone). CRH in 
turn causes the secretion of ACTH (adrenocorticotropic hormone) from the anterior 
pituitary. Finally, ACTH causes the secretion of cortisol from the adrenal glands 
[104]. The secretion of cortisol from the adrenal glands has a negative feedback 
effect suppressing the further secretion of CRH and ACTH [105, 106]. Between 15 
and 22 cortisol secretory events typically occur each day in a healthy adult [38, 107]. 
When cortisol is secreted into the bloodstream, a large percentage of it remains 
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bound [108]. It is the unbound cortisol in the blood that remains physiologically 
active [109]. This active cortisol aids in energy production at the liver [110, 111]. 
Since the cortisol concentration in the blood is a continuous variable and its pulsatile 
secretion forms an MPP, a state-space model for estimating the energy production 
level related to it should incorporate these types of observations. Similar to the case 
of skin conductance, a deconvolution procedure can be used to extract the pulsatile 
profile underlying a series of blood cortisol measurements [107]. Deconvolution 
also typically yields the infusion and clearance rates necessary to reconstruct a 
minute-by-minute profile of the cortisol concentration in the blood. Figure 8.1 shows 
a deconvolved cortisol profile [113]. 

Alternately, the same MPP plus continuous variable formulation can also 
be applied to skin conductance. Recall that skin conductance contains both a 
fast-varying phasic component and a slow-varying tonic component. The phasic 
component consists of a series of SCRs that are generated by neural impulses. 
These neural impulses form an MPP. The tonic component, which also reflects 
sympathetic arousal information, is a continuous observation [60]. Consequently, 
the state-space model with an MPP and a continuous observation can also be 
applied to the case of skin conductance. Unlike the case where we had one binary 
observation and two continuous observations to estimate sympathetic arousal from 
the same information, the formulation with the MPP and the continuous observation 
conforms more intuitively to the data itself. 
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Fig. 8.1 A deconvolved cortisol profile. Cortisol is secreted in pulses and between 15 and 
22 secretory events occur each day in a healthy adult. The figure depicts the blood cortisol 
measurements taken at 10 min intervals (blue), the reconstructed blood cortisol concentrations 
at a 1 min resolution (black), and the pulsatile secretions (red). From [112], used under Creative 
Commons CC-BY license 
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8.1 Deriving the Update Equations in the State Estimation 
Step 


Here again we will assume that xy varies with time following one of the state 
equations we have already seen. Therefore, no new predict step equations need to 
be derived. 

We made an interesting observation in the previous chapter when deriving 
the update step equations for the case where x, gives rise to MPP observations. 
We observed that the update equations switched between those where one binary 
variable was observed and where both a binary variable and a continuous variable 
were observed. We will now consider the case where we observe an MPP variable 
along with a continuous variable. As in (7.9), the density function for our MPP is 
still 


1 — pp = elog1.-po ifn, =0 
PNKA rE xe) = ; Org ro 10? ; -0k -0-71 79)? 
> se 205 = elg) e 205 if ny — 1, 
Pk y 2102 y 210? k 
(8.1) 


where ng and rg denote the occurrence of the point process events and the mark 
amplitudes, respectively. In addition to the MPP, we will now assume that we also 
Observe a continuous variable są where 


Sk = 80 + Ó1Xk + Wk, (8.2) 
and ôo, $1, and wg have their usual meanings. We observe s; at every point in time. 


Let us now proceed with deriving the mean and variance for the case when nz = 0. 
The posterior density in this case is 


2 
= (54-89-51 x4)” mulus ud 
= fae Uv ES 2 
TI xe 2w xe a, (8.3) 


Taking the log on both sides, we have 


(sk — 80 — 8p)? (Xk — xia? 
2 2 
20; 20 ka 


qi — log(1 — px) + constant. (8.4) 


We will next take the first and second derivatives of qı to obtain the mean and 
variance. 


dq 1 81(5k — ôo —Ó1Xx) (Xk — Xkik-1) 
2 - pk — po + l QUT eng 
dx (1 — px) o2 Okiki 


(8.5) 
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ôi (sx — ôo — 81Xx) (Xk — Xk]k-1) 
Pk + : 


=> 0= (8.6) 
2 2 
Ow Okik-1 
Since ng = 0, we can rewrite — py as (nk — px). Therefore, 
ô1(sk — ôo — ô1xXk) (Xk — Xkjk—1) 
nk — pr + 5 = AE (8.7) 
Ow Okik-1 


But this is identical to (4.21) with sx, $9, and $1 appearing in the equation instead of 
rk, Yo, and yı. Therefore, similar to (4.26), the update for the mean is 


2 
DUL 1 


Xk = Xkk-1 + [ozon pk) + 61 (sk — ôo — 81xux=1)]. (8.8) 


2-2 2 
0x1 + Ow 
We next take the second derivative of q1. 


dq 
dx? 7 


1 


82 
PKU — po — E (8.9) 
w 


gm 
Oklk—1 


This also happens to be identical to (4.28) but with à; and o2 instead of y; and o2. 
Therefore, similar to (4.29), the variance update is 


d? 1 2 1 82 =l 
kik = -( a ) = | 3 + Pkk(l — prix) + a . (8.10) 
dx, OK k-1 Oy 


This is interesting. When we observe both an MPP variable and a continuous 
variable and ng = 0, the update equations are identical to the case where one binary 
variable and one continuous variable were observed. 
We will next consider the case where nt = 1 and a non-zero mark r; exists. In 
this case, the posterior density is 
a PLT (sg 89810? DONET 
pary x eP909 xe eK ki . (81D 


Taking the log value and proceeding to take the first derivative, we have 


dq; 1 yiGk—yo—yixk) , 0163 — ôo — 81x). (Xk — Xkjk-1) 
=— px — po) 5 + a < 
dxy Pk o% Ow DUL 1 
(8.12) 
Vik — yo — Yaxk). , 15 — ĉo — ô1Xk) (Xk — Xkk-1) 
=1-—pet+ 2 + 2 3 | ; 
oy ow Tk 


(8.13) 
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Setting this to O and replacing (1 — px) with (ny — px) since nz = 1, we have 


dq» yı(rk — yo — yaxi). , 16s — ôo — 01x) (Xk — Xk]k-1) 
= nk — Pk + 2 + 2 2 = 0. 
AXk oí oi v1 
(8.14) 


But this is identical to (5.21) where we observed one binary variable and two 
continuous variables. The second derivative of q, yields 


2 
3? 1 


d? 2 
2 a (8.15) 
U 


= —pk(l — pr) 
2 2 2 
Ax; wo Ojk-1 


which is the same as (5.28). Therefore, in the case where we observe an MPP 
variable along with a continuous variable and ng = 1, our update equations in the 
state estimation step are identical to those where we have one binary variable and 
two continuous variables. 


When x, gives rise to MPP observations comprising of the pairs (nz, rg) and 
a continuous observation sz, the update equations in the state estimation step 
are 

if ny — 0, 


2 
DUL 1 
Xk = Xk|k-1 + 55 


— eie = pk) + y Gk — ĉo — bixu) | 
Yi 9g i + Ow 


(8.16) 
1 PI 
ik = l=— + pxik(l — Prix) + z| (8.17) 
0 O. 
k|k—1 w 
if ny = 1, 
2 
Okk—1 


Xk|k = Xk|k-1 + [o2z 0n — Pkik) 


DD 2 2.2 DD 
oio; + eiu 0 og + 9107) 


+ yo (rk — yo — Yixkik-1) + 9102 (sk — ôo — 5x1) | (8.18) 


2 24-1 
3 1 Y] ôi 
Ope =a — I Pr) | (8.19) 
Okk—1 Oy Oy 
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8.2 Deriving the Parameter Estimation Step Equations 


The derivation of the parameter estimation step updates is similar to what we 
have seen thus far. The updates for the parameters yo, yi, and o? related to rx 
are calculated based on the subset of values K corresponding to where n, = 1. 
The parameters ôo, 6), and o? corresponding to sz are calculated based on all the 
observations. 


8.3 MATLAB Examples 


The MATLAB code examples are contained in the folders shown below: 
* one mpp one cont 
sim\ 


data_one_mpp_one_cont.mat 
filter_one_mpp_one_cont.m 


expm\ 


expm_data_one_mpp_one_cont.mat 
expm_filter_one_mpp_one_cont.m 


The code itself is quite similar to what we have seen before in earlier examples. It 
takes in the inputs nx, rx, and sg denoted by the variables n, r, and s to estimate xx. 
We progress through the repeated predict, update, predict, update, etc. steps with 
Xkjk and og , being estimated using different equations based on nx. The variable 
names are also largely similar to what we have seen earlier. 


8.3.1 Application to Cortisol and Energy 


Recall the discussion regarding cortisol at the beginning of this chapter. Cortisol is 
secreted in pulses and between 15 and 22 of them are secreted by a healthy adult 
each day. The pulsatile hormone profile forms an MPP. In addition, the amount 
of unbound cortisol in the blood is biologically active and contributes to energy 
production. Thus, the observations for estimating the latent cortisol-related energy 
production state form an MPP and a continuous-valued variable. The cortisol inputs 
are provided to the code using the variables n, r, and s. The variables n and r 
denote the MPP observations n, and rg. The pulsatile secretions forming the MPP 
at a resolution of 1 min will need to be extracted via deconvolution (e.g., using [107, 
113, 114]). The cortisol infusion and clearance rates yielded by the deconvolution 
algorithm are used to generate sx. 
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Running the code examples for this particular state-space model produces the 
results in Fig. 8.2. The code running on experimental data for this model contains 
a notable difference. In general, when a continuous-valued observation is present, 
the state estimate can tend to overfit to it. In the experimental code example, the 
parameter estimation step updates for do, 51, and o? (the three parameters related to 
sx) have been adjusted so that only a small step is taken in the direction of the next 
predicted values at a time. A second change has also been made in that the sensor 
noise variance o2 is initialized at a larger value and the same three parameters óo, 
61, and o2 are only permitted to update until a reaches a threshold. These two 


changes greatly help reduce the overfitting to sx. 


State Estimation with Simulated Data State Estimation with Experimental Data 


= 10 U " 
9 Ss L € El eaen 
0 500 1000 1500 2000 2500 @& 5 . - ¿Mas qu A A A ENS eue. d om 
= . e ? U . 
v 
0.47 " N 
= 0.2 F 
= 20 
0 x - 
0 500 1000 1500 2000 2500 LÀ 
S 10 
2 
Lu 0 
S Duet ing, Lag e e gh] 
2 | | I l | 
0 500 1000 1500 2000 2500 0.04 


(d)x, 
o 


0 500 1000 1500 2000 2500 


5 
$ time index | 
z QQ Plot - State Estimate 219 
Cr x 
3 a e 
FI nl + S sl 
2 Es 4 > 
2, Í i | i | i | | 5 
s 4 3 2 -1 0 1 2 3 4 9 AM 1 PM 5PM 9 PM 1AM 5 AM 9 AM 
= standard normal quantiles time 


Fig. 8.2 State estimation based on observing one MPP and one continuous variable. The left sub- 
figure depicts the estimation on simulated data and the right sub-figure depicts the estimation 
of energy from blood cortisol data. The sub-panels on the left, respectively, depict (a) the MPP 
observations (blue) and the estimated rg (red), (b) the point process event occurrence probability 
Px (blue) and its estimate (red), (c) the continuous-valued variable s; (blue) and its estimate (red), 
(d) the state x, (blue) and its estimate (red), and (e) the QQ plot for the residual error of xg. The 
sub-panels on the right, respectively, depict (a) the deconvolved cortisol pulses (blue) and the fit 
to ry (red), (b) the reconstructed blood cortisol profile sg (orange) and its estimate (red), (c) the 
probability of pulse occurrence pz and its 95% confidence limits, and (d) the energy state x; and 
its 9596 confidence limits. From [33], used under Creative Commons CC-BY license 
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Chapter 9 A 
Additional Models and Derivations FEEN 


Much of what we have described in the preceding chapters provides the basic tools 
necessary to build physiological state-space estimators. In this chapter, we will 
briefly review some additional concepts in state-space estimation, a non-traditional 
method of estimation, and some supplementary models. These may help serve as 
pointers if extensions are to be built to the models already described. 


9.1 State-Space Model with a Time-Varying Process Noise 
Variance Based on a GARCH(p, q) Framework 


Thus far, we have not considered time-varying model parameters. In reality, the 
human body is not static. Instead it undergoes changes from time to time (e.g., 
due to disease conditions, adaptation to new environments). In this section, we will 
consider a state equation of the form 


Xk = Xk—1 + Ek (9.1) 


where e; ~ N (0, o? j)- Note that the process noise variance now depends on 
the time index k. Here we will use concepts from the Generalized Autoregressive 
Conditional Heteroskedasticity (GARCH) framework to model ex. In a general 
GARCH(p, q) framework, we take 


Ek = hve, (9.2) 
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where v; ~ N (0, 1) and 
q p 
h; = 00 + esL + X Bh... (9.3) 
i=l j=! 


where the o;'s and f;”s are coefficients to be determined. Now, conditioned on 
having observed all the sensor readings up to time index (k — 1), we have 


lel = E[Ag vy] = hElvk] = hy x 020 (9.4) 


and 


q p 

o2, = V (ek) = V (heve) = h3 V (vk) = hz x 1 = œo + RCM + 5h. 
i-l j=l 

(9.5) 


As is evident from (9.5), the variance of e, depends on k. If a GARCH(p, q) model 
is used for the process noise term in the random walk, the predict equations in the 
state estimation step change to 


Xk|k—1 = Xk-1k-1 (9.6) 


q p 
2 2 2 2 2 2 
Okjk-1 = Ok-ik-1i T Oek = Ok-ijk-1 T 60 + X QiEk—i + X Bjhy_j- OD 
i=1 j=l 


The update equations in the state estimation step remain unchanged. Note also that 
the calculation of og 1.1 requires the previous process noise terms. In general, these 
will have to be calculated based on successive differences between the x; and xķ—1 
estimates. 

Moreover, we would also have (p + q + 1) additional GARCH terms (the o's 
and f;’s) to determine at the parameter estimation step. These terms would have to 
be chosen to maximize the log-likelihood 


K 2 
(-D 1 (xk — Xk-1) 
0= >) logra?) + aan (9.8) 
k=1 esk 
(y € L a A2 
= x | log E (as + ae + et)! 
k=1 i-l j=l 
(xe — xia»? | (9.9) 
ao + S Aiek T Di Bjh; j 
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The maximization of Q with respect to the GARCH terms is rather complicated. 
Choosing a GARCH(1, 1) model for ez simplifies the computations somewhat. 
Additionally, note the recursive form contained within Q. For each value of k, we 
have terms of the form po . which contain within them further h? terms. In general, 
computing Q is challenging unless further simplifying assumptions are made. 


When x; evolves with time following xx = xx-1 + €, where ex is modeled 
using a GARCH(p, q) framework, the predict equations in the state estimation 
step are 


Xk|k—1 = Xk—1|k—1 (9.10) 
q p 
E er ¡he (9.11) 
Okjk-1 = Of 11 %o+ > we; t Bj k-j 
i=] y=1 


The parameter estimation step updates for the (p + q + 1) GARCH terms are 
chosen to maximize 


ED y Ñ z 
n | tog EON T 22773] 
k=1 im "ecl 
(xk — xii»? | 
ao + Diz wee; + 25 Bjh; j 


(9.12) 


9.2 Deriving the Parameter Estimation Step Equations for 
Terms Related to a Binary Observation 


Thus far, we have only considered cases where the probability of binary event 
occurrence px is of the form 


1 


E (9.13) 


Pk = 
We have also thus far only estimated £o empirically (e.g., based on the average 


probability of point process event occurrence). Occasionally, however, we will find 
it helpful to model pz as 


1 


1 + e- (Bo+B1xx) (9.14) 


Pk = 
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and determine fo and 6 at the parameter estimation step. If we wish to do so, we 
will need to consider the probability term that needs to be maximized at this step. 
Based on (3.27), this probability term is 


K K A : 1 
IT: nx lo - TI UIS +B1xx)+l08g ( Teo FPIK ) i (9.15) 


k=1 k=l 


This yields the expected log-likelihood 


K 
e=) [nio + fixi) — log (1 + ¿hosts | (9.16) 


k=1 


As in the case of determining the parameter updates for the terms in a CIF, this 
expected value is also somewhat complicated. Again, the trick is to perform a Taylor 
expansion around the mean RLY) = xx, for each of the individual log terms. After 
performing this expansion, we end up with terms like E[x, — xxx] and Ef (xk — 
Xk| x^] which greatly simplify our calculations. 

Let us begin by performing a Taylor expansion of the log term around xx |x [6]. 


log (1+ gu & log (1 + ePot Pix) + By PRK (Xk — Xgk) 
B 
En S Puk Ol — piyK ck — xk)”. (9.17) 


Note the terms (xx — xyjk) and (xy — Xx K) in the expansion. Taking the expected 
value on both sides, 


| log (1 + buns) = log (1 + ePot Pix) + Bipeix [o EM xk | 


Bi , 
+ 5 Puk U — pai) EL (xe — xiuk?] (9.18) 
Bo--Pixk|k Bi 2 
=108 (1+e K)+0+ 5 Pak — Pri K DOG K- 


(9.19) 
Therefore, 
K 07 
Qm 3 [mh + Pixx) — log (1 + eue) — z Puk U = undo | 


k=1 
(9.20) 
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Now, 
OPK _ 9 | 1 | " (=1) s [ bot Biman) 
dBo du | 1 + e- Bo+Brxeik) E D: a 
= puk(d — prix). (9.21) 
And similarly, 
Opk|K 
= PKC — Pk|k)Xkk- (9.22) 
0061 


Taking the partial derivative of O with respect to Bo, we have 


K 2.5 
00 | eo Bi xix Pi a 
K 2.2 
Bion 3 
= nc- pak prik (1 — prix) | (9.24) 
E | 2 val | 
K 2.2 
Bro 
= X |n DK : ~ (1 — pyx)d — Zu pu | (9.25) 
k=1 


And similarly for 81, we have 


K 


2 
0 O 
Say =) LT —Xk|K Pk|K > prix (.— pax) [2 + Bixiak (1 — uo] 
k=1 
(9.26) 
By setting 
ð 
3 _9 (9.27) 
dBo 
0 
99 nd (9.28) 
981 


we obtain two simultaneous equations with which to solve for Bo and 81. Note also 
that the use of 6o and f, in pz causes changes to the filter update equations for xy 
and og ke 
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The parameter estimation step updates for fp and £j when we observe a 
binary variable nz are obtained by solving 


K M 


Bi OK 
> DUO (1 — puk)( — 2px¡k)px¡xk | =0 (9.29) 
k=1 


K 


Dig x 
Y [nixuk — xk Perk — z PRK Pak) [2 + Bixex (1 — 2pxx)] 
k=1 


=0. (9.30) 


9.3 Extending Estimation to a Vector-Valued State 


We have also thus far only considered cases where a single state x, gives rise to 
different observations. In a number of applications, we will encounter the need to 
estimate a vector-valued state xg. For instance, we may need to estimate the position 
of a small animal on a 2D plane from neural spiking observations or may need to 
estimate different aspects of emotion from physiological signal features. We have a 
multi-dimensional x; in each of these cases. 

Let us first consider the predict equations in the state estimation step. Assume 
that we have a state x; that varies with time following 


Xy = AXx-1 + Buy + e, (9.31) 


where A and B are matrices and ey ~ N(0, X) is the process noise. The basic 
statistical results related to mean and variance in (2.1)-(2.6) simply generalize to 
the vector case. Thus, the predict equations in the state estimation step become 


Xkjk—1 = AXk-i1jk-1 + Buy (9.32) 
Xkk-1 = AZXk-ik-iAT + X, (9.33) 


where the covariance (uncertainty) X of xy is now a matrix. 

Recall also how we derived the update equations in the state estimation step. 
We calculated the terms that appeared in posterior p(xk|y1:x) and made a Gaussian 
approximation to it in order to derive the mean and variance updates xp¡x and og p 
In all of the scalar cases, the log posterior density had the form 
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(Xk — xii? 


+ constant, (9.34) 
2 
2040.4 


qs = f (Xk) — 


where f (xx) was some function of xg. This function could take on different forms 
depending on whether binary, continuous, or spiking-type observations (or different 
combinations of them) were present. In each of the cases, the mean and variance 
were derived based on the first and second derivatives of qs. 

There are two different ways for calculating the update step equations in the 
vector case. 


* The first is the traditional approach outlined in [10]. Here, the result that holds 
for the 1D case is simply extended to the vector case. Regardless of the types of 
observations (features) that are present in the state-space model, the log posterior 
is of the form 


1 E 
qv = f 3k) — ¿(Xt = SKIT Egg. (Kk — Xk\k—1) + constant. (9.35) 


The manner in which the updates xy, and Zu are calculated, however, is quite 
similar. We simply take the first vector derivative of g, and solve for where it 
is 0 to obtain xy. We next take the Hessian of q, comprising all the second 
derivatives and take its negative inverse to obtain Zi. 

* The second approach is slightly different [115]. Note that, based on making a 
Gaussian approximation to the log posterior, we can write 


1 7 1 Ñ 
= 5 OU Xi) Y Eg Qu — xo = F(x) 5 0 — Xie)" E (Xi — Xkpj-1) 
+ constant. (9.36) 


Let us take the first vector derivative with respect to x on both sides. This yields 


E Of (Xx) E 
_ Xi 8 — Xen) = a Dae Fe — Xik-1)- (9.37) 
Let us now evaluate this expression at Xy = Xyjy—1. Do you see that if we 


substitute x; = X,\x—-1 in the above expression, the second term on the right 
simply goes away? Therefore, we end up with 


= Of (xx) 
— Exp (Ree 1 — Xe) = Í S (9.38) 
Xk Xk\k—1 
Of (xx) 
=> Xk = KKL + Lele a (9.39) 


Xk|k—1 
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This yields the mean state update for xi. How do we derive the covariance 
matrix Xk? We simply take the vector derivative of (9.37) again. Note that in 


: 2, : oe : 
this case, 27 is a matrix of all the second derivative terms. Thus, we obtain 
k 


- 3? f (x) i 
1 1 
Ek == Jx + Xa (9.40) 
k 
f) a] 
=> Sy =|- = + zaha l (9.41) 
9X; 


9.4 The Use of Machine Learning Methods for State 
Estimation 


Machine learning approaches can also be used for state estimation (e.g., [116, 117]). 
In these methods, neural networks or other techniques are utilized to learn a 
particular state-space model and infer the unobserved state(s) from a dataset. In this 
section, we will briefly describe how the neural network approach in [116] is used 
for estimation. In [116], Krishnan et al. considered the general Gaussian state-space 
model 


xp ~ N (Sux Qs faa 1) (9.42) 
yk ~ H Cfy Gi), (9.43) 


where y, represents the observations. Both the state equation and the output 
equation are learned using two separate neural networks (for simplicity, we group 
both of them together under the title “state-space neural network"—SSNN). A 
separate recurrent neural network (RNN) is used to estimate xg. Taking y and @ 
to denote the parameters of the state-space model and the RNN, respectively, the 
networks are trained by maximizing 


K 
Q = YE, sus [log py Oxlxi)] 7 KL(qoGalY)llpy Gu) 


K 
— MES (p11) [KL (96 bui. 3) py Gr lx). (9.44) 


where py(-) and qg(-) denote density functions [116]. The actual training is 
performed within the algorithm as a minimization of the negative term which we 
label Qr. Analogous to the state-space EM algorithms we have seen so far, in 
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this neural network approach, the SSNN replaces the explicit state-space model, the 
RNN replaces the Bayesian filter, and the weights of the neural networks replace the 
model parameters. The objective, however, is still to estimate xy from observations 
such as ng, rg, and sz. Since neural networks are used to learn the state-space model, 
more complicated state transitions and input-output relationships are permitted. One 
of the drawbacks, however, is that a certain degree of interpretability is lost. 

Similarities also exist between the terms in Oy and the log-likelihood terms 
we have seen thus far. For instance, when a binary variable ng is present among the 
observations yx, Om contains the summation 


=y |n. log (+ + Lw) + (1 — ny log (1 = —39J' (9.45) 


Take a moment to look back at how (3.15) and (3.26) fit in with this summation. 
In this case, however, f,(-) is learned by the SSNN (in our other approaches, we 
explicitly modeled the relationship between xy and pz using a sigmoid). Similarly, 
if a continuous-valued variable s; is present in yz, there is the summation 


[sk — fu, Gp 


1 
y = log [277 f,2(x4)] + Df ae > 


> (9.46) 
where f,,(-) and f,2(-) represent mean and variance functions learned by the 
SSNN. Again, recall that we had a very similar term at the parameter estimation 
step for a continuous variable sz. 

One of the primary advantages of the neural network approach in [116] is that 
we no longer need to derive all the EM algorithm equations when new observations 
are added. This is a notable drawback with the traditional EM approach. Moreover, 
we can also modify the objective function to 


Q0 — p)QuL +0 DY 10, (9.47) 


where lL is an external influence and 0 < o < 1. This provides the option to perform 
state estimation while permitting an external influence (e.g., domain knowledge or 
subject-provided labels) to affect xg. 


9.5 Additional MATLAB Code Examples 


In this section we briefly describe the two state-space models in [118] and [30] 
for which the MATLAB code examples are provided. The equation derivations for 
these two models require no significant new knowledge. The first of these incor- 
porates one binary observation from skin conductance and one EKG spiking-type 
observation. The second incorporates one binary observation and two continuous 
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observations. It is almost identical to the model with the same observations 
described in an earlier chapter but has a circadian rhythm term as 7%. The derivation 
of the state and parameter estimation equations is similar to what we have seen 
before. 


9.5.1 State-Space Model with One Binary and One 
Spiking-Type Observation 


The MATLAB code example for the state-space model with one binary and one 
spiking-type observation is provided in the "one bin, one spk" folder. The model is 
described in [118] and attempts to estimate sympathetic arousal from binary-valued 
SCRs and EKG R-peaks (the RR-intervals are modeled using an HDIG-based CIF). 
The results are shown in Fig. 9.1. The data come from the study described in [119] 
where subjects had to perform office work-like tasks under different conditions. In 
the first condition, the subjects were permitted to take as much time as they liked. 
The other two conditions involved e-mail interruptions and time constraints. Based 
on the results reported in [118], it appeared that task uncertainty (i.e., how new the 
task is) seemed to have generated the highest sympathetic arousal responses for the 
subject considered. 


9.5.2 State-Space Model with One Binary and Two Continuous 
Observations with a Circadian Input in the State 
Equation 


Cortisol is known to exhibit circadian variation [120, 121]. Typically, cortisol 
concentrations in the blood begin to rise early morning during late stages of sleep. 
Peak values are reached shortly after awakening. Later in the day, cortisol levels 
tend to drop toward bedtime and usually reach their lowest values in the middle of 
the night [122, 123]. In [30], a circadian J, term was assumed to drive x; so that it 
evolved with time following 


Xk = PXk-1 + Ik + £k, (9.48) 
where 
2 2rrik 2rrik 
Ik = Xa sin (=) + bj COS (uo) (9.49) 


The model also considered the upper and lower envelopes of the blood cortisol 
concentrations as the two continuous variables rą and sg. The pulsatile secretions 


9.5 Additional MATLAB Code Examples 107 


State Estimation with Experimental Data 


4 
<x 
I 95 
e 
0 
0 10 20 30 40 50 


time (min) 


Fig. 9.1 State estimation based on observing one binary and one spiking-type variable. The sub- 
panels, respectively, depict (a) the skin conductance signal zz (the green and black dots on top 
depict the presence or the absence of SCRs, respectively), (b) the RR-interval sequence (orange) 
and the fit to the HDIG mean (red), (c) the probability of SCR occurrence pz and its 95% 
confidence limits, (d) the arousal state x, and its 95% confidence limits, and (e) the HAI (the 
regions above 90% and below 10% are shaded in red and green, respectively). © 2019 IEEE. 
Reprinted, with permission, from [118] 


formed the binary variable nz. The inclusion of each continuous variable neces- 
sitates the determination of three model parameters (two governing the linear fit 
and the third being the sensor noise variance). In addition, the state-space model 
in [30] also estimated fp and £4 in px. There are also six more parameters in the 


108 9 Additional Models and Derivations 


State Estimation with Experimental Data 


o 
o 
m4 
œ 
T 


(d) p 


0000 1200 0000 1200 0000 1200 0000 1200 0000 1200 
time (24h clock) 


Fig. 9.2 State estimation based on observing one binary and two continuous variables with a 
circadian input in the state equation. The sub-panels, respectively, depict (a) the cortisol profile (the 
green and black dots on top denote the presence or the absence of pulsatile secretions respectively), 
(b) the first cortisol concentration envelope rg (green solid) and its estimate (dashed), (c) the second 
cortisol concentration envelope s, (mauve solid) and its estimate (dashed), (d) the probability 
of pulse occurrence py, and (e) the energy state xx. © 2019 IEEE. Reprinted, with permission, 
from [30] 


state equation: p, a4, a2, b1, b», and m. To ease computational complexity, the 
EM algorithm in [30] treated the four parameters related to the circadian rhythm 
(aj, a», by, and b2) somewhat differently. Thus, while all the parameters were 
updated at the parameter estimation step, a1, a2, b1, and b» were excluded from the 
convergence criteria. The results are shown in Fig. 9.2. Here, the data were simulated 
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for a hypothetical patient suffering from a type of hypercortisolism (Cushing’s 
disease) based on the parameters in [124]. Cushing’s disease involves excess cortisol 
secretion into the bloodstream and may be caused by tumors or prolonged drug use 
[125]. Symptoms of Cushing’s disease involve a range of physical and psychological 
symptoms including insomnia and fatigue [126—128]. The resulting cortisol-related 
energy state estimates do not have the usual circadian-like patterns seen for a healthy 
subject. This may partially account for why Cushing’s patients experience daytime 
bouts of fatigue and nighttime sleeping difficulties. 
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Chapter 10 
MATLAB Code Examples 


10.1 State-space Model with One Binary Observation 


10.1.1 Simulated Data Example 


load('data one bin.mat'); 


K = length(n) ; 
M = 2e4; 
ve = zeros(1, M); $ process noise variance 


be jasa = eros ll, 19) 7 
v_pred = zeros(1, K); 


ES urea = meras (il, EX) 4 
m owoce = zeroa; p 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


pEUpdES-Eeros MEHR 


base prob - sum(n) / length(n); 


tol = 1e-6; $ convergence criteria 
A - zeros(1, K); 

W - zeros(1, K); 

CW - zeros(1, K); 

(G c ron, 187 

ve(1) = 0.005; 

x smth(1) = 0; 


bo = log(base prob / (1 - base prob)); 


© The Author(s) 2024 
D. S. Wickramasuriya, R. T. Faghih, Bayesian Filter Design for Computational 
Medicine, https://doi.org/10.1007/978-3-031-47104-9 10 


® 
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if (k == 1) % boundary condition 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 


ao) = pe mole (ls > al) g 
= Weiwlheche (ie = db) = Aven) p 


< 
la 
K 
0) 
Q 
& 
I 


x updt(k) = get state update(x pred(k), v_pred(k), bO, n( 


KI 
p updt(k) = 1 / (X + exp((-1) * (bO + x updt(k)))); 
v updt(k) = 1 / ((1 / v pred(k)) + p updt(k) « (1 - 
p updt (k))); 
end 
He Ema RE corp d (ES) $ 
v smth(K) - v updt(K); 
W(K) = v smth(K) + (x smth(K) ^ 2); 
A(1:(end - 1)) = v updt(1:(end - 1)) ./ v pred(2:end) ; 
for k= Uc Il 
x smth(k) = x updt(k) + A(k) « (x smth(k + 1) - x pred(k + 
1) 5 


v_smth(k) = v updt(k) + (A(k) ^ 2) « (v smth(k + 1) - 
v pred(k + 1)); 


CW(k) = A(k) « v smth(k + 1) + x smth(k) « x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 


if (m « M) 


ve(m + 1) = (sum(W(2:end)) + sum(W(1:(end - 1))) - 2 x 
sum(CW)) / K; 
mean dev = mean(abs(ve(m + 1) - ve(m))); 


if mean dev « tol 
fprint eum cu TT cn ky, USE aa) tin, 
Y) SO 
fprintf ('Converged at m = %d\n\n’, m); 
break; 
else 
fprint e Gmi Scale Sey, Ilse Nia el?) ti, 
x smth(1), ve(m + 1)); 


x pred - zeros(1, K); 
v pred zeros(1, K); 


10.1 State-space Model with One Binary Observation 


se wüselE c vemos (al, 19) 7 
y tos = moro (al, 10) y 


sie fca acc) c meros, mK = dd), $ x smth(1) 
needed for next iteration 
v ema S Mero PEKI 


B toci = zeros; p 


A = zeros(1, K); 
W = zeros(1, K); 
Ig 

K) ; 


r 


CW = zeros (1, 
C = zeros (i, 


end 


p smth = 1 ./ (1 + exp((-1) + (bO + x_smth))); 


figure; 
subplot (411); 
stemin, “Elli”, “color”, [070.75 0105 


Milan EO) 1.251) 5 
ylabel (' (a) n (k)'); 
grid; title('Estimation with Simulated Data’); 


subplot (412); 


hold om; 

piloto bI 

pilotipmsmthy qe new dea Zo) 
ylabel('(b) p E 

grid; 


subplot (413); 


hold on; 

plot = EA 

alore mme r= p erase 125) p 
ylabel('(c) x (k)'); xlabel('time index’); 
grid; 


subplot (414); 

qqplot (x - x smth); 

title('00 Plot - State Estimate’, 'FontWeight', 'Normal'); 
ylabel('(d) input quantiles'); 

xlabel('standard normal quantiles'); 

grid; 


function [y] = get state update(x pred, v pred, b0, n) 
M = 50; $ maximum iterations 


it - zeros(1, M); 
func = zeros(1, M); 
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sb (d) = sm recl; 
ene al ex abg Qui = 3b) 
func(i) = it(i) - x pred - v pred « (n - exp(bO + it(i)) / 
(1 + exp(bO + it(i)))); 
he (a) = al <= Y jad ex crgo + eE) Y (Gl ss Ello + alte (al 
1) 2)5 
el s 30) s dela) = Ema (E) / Ge (al); 
if abs(it(i + 1) - it(i)) < 1e-14 
y = alice (al oss al) 5 
return 
end 


end 
error ('Newton-Raphson failed to converge.'); 


end 
10.1.2 Experimental Data Example 


load('expm data one bin.mat'); 


K 
n 


length (u) ; 
zeros(1, K); 


I 


joie cm crab > 10) 7 


n(pt) = 1; 
M = 2e4; 
ve = zeros(1, M); $ process noise variance 


ES oras! Ss erosi CPER 
v_pred = zeros(1, K); 


Be wcle = ewe. HE 
Wee = Mero (al, ER 


Pe mt = Luo ll, 1) y 
HZ femel cs zon PIRKE 


[o uoci = wero (al, 1) E 


base prob - sum(n) / length(n); 
tol = 1e-8; % convergence criteria 


AU co oxerosdi xe 
W = zeros(i, K); 
KE 
KOE 


1 


CW = zerosil, 
C 2 xeroeit, 


10.1 State-space Model with One Binary Observation 


ve(1) = 0.0 
x med ) = 


¡PU 
vu 


p updt (k 


end 


x smth(K 
v smth(K 


W(K) - 


A(1:(en 
x0 prev 


for kc 
x sm 
"E 


v sm 


05 


1) = s 
b0 = log(base prob / 


(1 - base prob)); 


% boundary condition 
= x smth(1); 
= ve(m) + ve(m); 


x pred(k) = x updt(k - 1); 
v pred(k) = v updt(k - 1) + ve(m); 


pdt(k) = 
pdt(k) = 
23) 8 


) c 22 Wwe 
VO 
v_smth(K) 


a e 1j) = 


(K - 1): 
th (k) 


aa) = wp 


v_pred(k + 1)); 


I 
x 
E 

tej 
Q 
e 


get state update(x pred(k), v pred(k), 


a / 
i (a / y ardido) x i9 metes) cv (di 


dt (K) ; 
dt (K) ; 
* (x smth(K) ^ 2); 


viupdt (1+ (end = 1))  / vipred(2-end); 
S mise 


updt(k) + (A(k) ^ 2) * (v smth(k + 1 


) 


bo, 


(1 + exp((-1) * (bO + x_updt(k)))); 


k) + A(k) * (x smth(k + 1) - x pred(k 
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n( 


+ 


< 


CW(k) = A(k) x v smth(k + 1) + x smth(k) « x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
end 
if (m « M) 
ve(m + 1) = (sum(W(2:end)) + sum(W(1:(end - 1))) - 2 + 
sum(CW) + 0.5 « W(1)) / (K + 1); 
530 > ss pua) / gs 
if (abs(ve(m + 1) - ve(m)) « tol) && (abs(xO - x0 prev) 
tol) 
fprine nim SENO c» Eyck abu = Bole aa y wy 
x smth(1), ve(m)); 
fprintf('Converged at m = %d\n\n’, m); 


break; 


116 
else 

more (in) = Exch ieee) 

x smth(1), ve(m + 1)); 
pre doo Sl 
Vapneda——Zzeros (ly, 
SGRUpGdEs= zeros al, 
y woche = RE EI 
x smth(2:end) = zeros(1, 

needed for next iteration 
x smth(l) = x0; 
v_smth = zeros(1, 
pPRUPAt = zeros Gy 
JN c3 Zeros PIRK, 
WE-SEZerosi( MK) 
(WES Zero SEE 
eS mero, 1) 7 

end 
end 
end 


p smth = 1 ./ (1 + exp((-1) + 


lcl x - norminv(0.025, x smth, 
Cel ss > moria (09/5: St: hy, 


certainty = 1 - normcaf (prctile(x_smth, 
x smth)), x smth, sqrt(v smth)); 


lcl p zenos( 195 
ucl p - zeros(1, K); 
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ES Bt mm» 


$ x smth(1) 


(bo + x smth))); 


sqrt(v smth)); 
sqrt(v smth)); 


* ones(1, length( 


disp('Calculating the pk confidence limits... (this can take time 


due to the resolution)'); 
Bor kom 


[lcl_p(k), ucl p(k)] = get pk conf lims(v smth(k), bO, x smth 


CEDE; 
end 


disp('Finished calculating the pk confidence limits.'); 


fs = 4; 
E e (080% = 3)» / Es, 
tr = ((K - 1):(-1):0) / £s; 


u plot = NaN » ones(1, K); 
u plot(pt) - u(pt); 


subplot (511); 
hold on; 
pISCEEV yc new tdt 


10.1 


ylabel((' (a) 
set (gca, 'xticklabel', 
TGT 


skin Gel 


NE 


State Estimation with 


State-space Model with One Binary Observation 


(imu S)'}); 
ylim([0 31); 


Experimental Data'); 
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xlim([0 ty(end) 


IDE 


grid; 

NAL = sdb 

parda Ea), s39(2); MA), sa) Y ses, ALA) wil (al) yal (a) se 
MAE racenlpha*, 02 Edge color none: 

EA), s9(9)65 a). a 4 ws, AA) sub(ab) walla) vale 
Me “Cs ‘racenipna’ , 0:2, "Edgecolor”, “none” )i; 

patch([xp(3), xp(4), xp(4), xp(3)] / fs, [yl(1) yl(1) yl(2) yl(2 
le I 0-627059 me 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

patch(Ixp(4), xp(5), xp(5), xp(4)] / fs, [yl(1) yl(1) yl(2) yl(2 
H us sta Ro ae, 0.2, “Edgecolor”, nonet); 

pacchia); MS). 24906), MS / is, ALA) Aa yaa) ae 
U nm pu e 02; Ed e colo E (ey nons) 

subplot (512); 

eee (ey E du UESLT^. f^. “markersize’, 3); 

ylabel( a heb. (eh) m cae salia (O dece) s salam © 315] ) 5 

hal = SE 5 on erecta pe 

arca (Esa), 39020) 5 T sio / es, NAL) A) yla) sem 
le H racehlpha*, 02, Edge color | none); 

tE) a A). Y Y x ID) wll) sale) yal 
Me Esta racehlpha*, 0:2, Edcgecolor o mons) 

patch(Ixp(3), xp(4), xp(4), xp(3)] / fs, [yl(1) yl(1) yl(2) y1(2 
le Li Osea 7059 s 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

patch(Ixp(4), xp(5), xp(5), xp(4)] / fs, [yl(1) yl(1) yl(2) yl(2 
la ens fare ee po 0.2, Edgecolor, nonet); 

pacchi (so), MS). 24906), MS) 7 is), BAG SA yaa) sem 
1 RT uc e d 02, EE de color Se none a) i, 

subplot (513) ; 

held on; 

plete, Se Ema, US, lineales, ib 25) p 

end le, ly leal ox Flia (mal Se) I), “6%, aereos”, "mena y Y 
FaceAlpha’, 0.5); 

ylabel('(c) state (x_{k})’); ylim([-10 51); 

set(gca,'xticklabel', []); xlim([0 t(end)]); 

gjana Will e sim 

peccal taati); x92); a), ER ff xen, IWAL) svi (al) atq) sa 
0 agi, L 072) HE TEE [E nonet); 

pare) US), IT / x (lab) wb (Gl) sal) vaa 
ie epee 'racenlpha^, Vea) ndgeColon im Honc 

patch(Ixp(3), xp(4), xp(4), xp(3)] / £s, [yl(1) yl(1) yl(2) yl(2 
le If 0.647059 a. 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

patch(Ixp(4), xp(5), xp(5), xp(4)] / £s, [yl(1) yl(1) yl(2) yl(2 
Jost (racehlpha*, Or2 Edge colo m nons 

¡rca ES), MS), O), MY 77 sesh, (yal (ab) AE aa) ae 
le Me ee 0:2, EddgeCcolor i ‘nonet; 
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subplot (514); 


hold on; 

plott 19 Ema, “2%, Cabiiaeeheurim s 3LCINE 

cade, erally, Hel jer aake joy, (lil, O, (ley / 99S) © 
EdgeColor', ‘none’, ‘“FaceAlpha’, 0.3); 


a O BEH E 

ylabel('(d) probability (p (k]))') 
set(gca,'xticklabel', []); xlim([0 t(end)]); 
Gaele yal e wu 


parc tsali); 269(2), sso(23)5 si f/f xe Lab) wil (al) walla) se 
Tee 'racenlpha^, 0.2, “Hdgecollon” “none ji 

¡rc bala); sq9(9)5 A). A 4 a) AA) will) walle) sale 
Va CIE 'racenlpha^, 042: "mies", "usus 

patch(Ixp(3), xp(4), xp(4), xp(3)] / £s, [yl(1) yl(1) yl(2) yl(2 
i, [2 0.547059 a, 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

patch([xp(4), xp(5), xp(5), xp(4)] / £s, [yl(1) yl(1) y1(2) yl(2 
jen er (racehlpha*, 0727  EdcgeCcolor'7 “none” )); 

are), (6); ME), MSI Y ses, [yl (at) sil (i) ye) sae 
M odi fo R 0ec2 mU EEdgecolor “nonet; 


subplot. (Sais) 

hold on; 

wh = [0 0.9. tíend) 0. 

eu = qub (220 / 255) Y 
Op al (9) 017 

Facesi = [2 2 3 4I5 


9; t(end) = () dl p 
220 / 255), 1 (220 / 255) (220 / 265); 1 0 


patch('Faces', facesl, 'Vertices', vl, 'FaceVertexCData', cl, ' 
FaceColor', 'interp' ; 
'EdgeColor', 'none', 'FaceAlpha', 0.7); 


v2 [DO 0; Elena)! 0; tlend) Osi; 0 0.115 

G2 = [© 0,3 07 0 0.3 07 (oa / 255) 1 (204 7 255), (204 / 255) 3 
(204 / 255)]; 

faces2 = [1 2 3 415 


Il 


patch('Faces', faces2, 'Vertices', v2, 'FaceVertexCData', c2, ' 


FaceColor', 'interp' , 
"BdgeColor'7 “none”, "FaceAlpha' 0-7); 
alce (E, csrralmey, "cues", Misa / 255) (43 / 255) (226 / S55 
mn ewrata I5); gridi, 
ylabel('(d) HAI’); xlabel (time (s)'); xlim([0 t(end)]); 


function [y] = get state update(x pred, v pred, b0, n) 


MESE % maximum iterations 
ae > esos M) 
func = zerosil, M); 


df = zeros(1, M); 


shed) = x pene 


10.2 State-space Model with One Binary and One Continuous Observation 


for 2 JM = 4 
fune (a) 
(1 + exp(b0 + it(1)))); 
df(i) = 
03) ^ 2»5 
sah se 3D) == sue) = asked) / hs 


ae Els abeo ab gs dl) > ae ((ab)))) cem issid 
y = dei < 3) 5 
return 
end 
end 
error ('Newton-Raphson failed to converge.'); 
end 
ievinere sera, lel, vell = ger jos senta limat, 190, ES) 


p = (1e-6:1e-6:1); 


is = Comecon Gon d. jf leerla c qnn xs wÀ) xy 19 ac 


EMS) 7 (2 c we Cesa of (b =D) & ess) )) 


J^ By) 5 


ig ex melo Es 0.3975) y 
ma anclas e 0.025) F 


wel = jal 


p(n(end)); 
lel = p(m(en 


e» 
end 


10.2. State-space Model with One Binary and One 
Continuous Observation 


10.2.1 Simulated Data Example 
load('data one bin one cont.mat'); 


K = length(n) ; 


joue e arbori om. m» (99) y 


M = 5e4; 
ve - zeros(1, M); $ process noise variance 
r0 = zeros(1, M); % linear model coefficients (continuous 


variable) 


(1 - p) 
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= it(i) - x pred - v pred « (n - exp(bO + it(i)) / 


1 + v pred + exp(b0 + it(i)) / ((1 + exp(bO + it(i 


~* 


- x) 
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ial cr weirs (il, Mg % linear model coefficients (continuous 
variable) 
vr - zeros(1, M); % sensor noise variance (continuous variable) 


pel >= amos C PERKE 
v_pred = zeros(1, K); 


Pe ole cx amos, E y 
z uocis = Emos ll, 19) y 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


[o uocis = EZ STO SEE 


base prob - sum(n) / length(n); 
tol = 1e-8; % convergence criteria 
A - zeros(1, K); 
W = zeros(i, K); 
CW = zeros(1, K); 
Qo zero, Kis 


1 


a / (1 - base_prob)); 


if (k == 1) 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
cdo) = ox ioe (ie = aly 
v_pred(k) = v_updt(k - 1) + ve(m); 
end 
x updt(k) = get posterior mode(x pred(k), v pred(k), r(k) 
(iin) » va (um), 190, sas) 


p updt(k) = 1 / (1 + exp((-1) + (bO + x_updt(k)))); 

v updt(k) = 1 / ((1 / v_pred(k)) + ((ri(m) ^ 2) / vr(m)) 
H pupat (K) * (1 = plupdt (k))05 
end 


x smth(K) = x updt(K); 
v smth(K) - v updt(K); 
W(K) = v smth(K) + (x smth(K) ^ 2); 


A(1:(end - 1)) = v_updt (1: (end - 1)) ./ v pred(2:end) ; 


10.2 State-space Model with One Binary and One Continuous Observation 
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for koc UK pibe er 
x smth(k) = x updt(k) + A(k) « (x smth(k + 1) - x pred(k + 
IDDE 
v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 
v pred(k + 1)); 
CW(k) = A(k) + v smth(k + 1) + x smth(k) + x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
end 
if (m « M) 
R - get linear parameters(x smth, W, r); 
rOüun «4 1) = Rit, Ll; 
ial (a <b 30) = QE, 1) 
vr(m + 1) = get maximum variance(r, rO(m + 1), rl(m + 1), 
W, x smth); 
ve(m + 1) = (sum(W(2:end)) + sum(W(1: (end - 1))) - 2 x 
sum(CW)) / K; 
mean dev = mean(abs([ve(m + 1) rO(m + 1) r1(m + 1) vr(m + 
3] = [el oy sesh (n) pea Qn) 1): 
if mean dev « tol 
fprintt (A RENO olaaa e E SE ICI HE 
ars cs fles enu: e Soleras, mm) we Suela (Gb), 20m), mL), we 
(m), ve(m)); 
fprintf ('Converged at m = %d\n\n’, m); 
break; 
else 
pcmcia (“ji = ENE Ex Edo cs ey ase Viewed, cs fy AE 
Nowa = ae =) 3, daft nno ms x Ema), rom ss 20) em 
i i), mea & 2), wai c 10) 
se joel c» aros (a, HQ 7 
v pred = zeros(1, K); 
se Wheels c raros (al, 10) y 
W whole = meros, 19) 7 
amen endi menos (al, i = 35 $ x_smth(1) 
needed for next iteration 
y amda = egos (al, 1%) 7 
Jo Whole = Aros (il, 19 E 
¡ZO SK 
We massa, 17 
CW = zeros(1, K); 
(Ge reas lll, X9 E 


end 
end 
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end 

winds e 1 of (Gb s codel) s (00 | ss _wisele))) 5 
p smth = 1 ./ (1 + exp((-1) + (b0 + x smth))); 
r smth = rO(m) + rl(m) + x_smth; 

figure; 

subplot (511) ; 

aaen, “elie, "Use". (10 Os Ol 


ial ORE 5] a 
ylabel('(a) n (k)'); 
grid; title('Estimation with Simulated Data'); 


subplot (512); 


hold on; 

¡lors 197) 

loe (e mme Um, y, “ilsiaenjalelelaY s dbs 9 7 
viable 

gula! 


subplot (513); 


hold on; 

pilot E 

ploUüpEsmth Mr Eam ewe , abeS)) 9 
ylabel('(c) p (k)'); 

grid; 


subplot (514); 


hold on; 
piloto mI) 
[okeje (es uncle, Ure. Vabiscivhüeheims , alo Sy) y 


ylabel( (d) xe 
xlabel('time index’); 
grid; 


subplot (515); 

qqplot(x - x smth); 

title('00 Plot - State Estimate’, 'FontWeight', 'Normal'); 
ylabel('(e) input quantiles'); 

xlabel('standard normal quantiles'); 


grid; 

piero (Al TIE ES lor mole edere Vpred ATR sail, Was, 
50, m) 
Moz 1007 % maximum iterations 


he = zeros COMEM) p 
f = zeros(1, M); 
df - zeros(1, M); 


aig (GL) 


x pred; 


for i = 1:(M - 1) 


10.2 State-space Model with One Binary and One Continuous Observation 


end 


func 


end 


tune 


end 


123 


Seca (bei 2) ya sr sas) E 

iE GL) c ake (GL) = oe pued = E 5 (alos (es = s = asd e ss joel) 
ap Mae rs (Gay a abo» Lo OLEA ES NA 

cl2 (st) e 1 © c sam pj + se(G) Y UA. 00 c sed 
315)) ° Byes 

ais (GL s 3) — e del(a) = ss) / CE) A 

sb close (a <> 3)) = shie((at))) cm dien 


y LE es a 
return 
end 


end 


error (’Newton-Raphson failed to converge 


tion y = get maximum variance(z, r0, rl, 

K - length(x smth); 

x e (uc 2 is loc (60 ^ By & (uu ^ ES o 
- 2 x rO sum(z) - 2 x rl x 

r0 * rl * sum(x smth)) / K; 

tion y - get linear parameters(x smth, W 

K - length(x smth); 

y = [K sum(x smth); sum(x smth) sum(W)] 

x smth)]; 


10.2.2 Experimental Data Example 


load('expm data one bin one cont.mat'); 


K 


length (n); 
ml > ODE 
5e4; 
zeros(1, M); $ process noise variance 
zeros(1, M); $ linear model coefficie 
variable) 
zeros(1, M); % linear model coefficie 
variable) 
zeros(1, M); % sensor noise variance 


sg 


W, x smth) 


sum (W) 


got Sinica, +2) CE ME 


z) 


D 


X [sum(z) : sum(z .* 


nts (continuous 


nts (continuous 


(continuous variable) 
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x-EDredE-Ezeros MERE 
v pred - zeros(1, K); 


be Morla zeros (1, 19) 7 
tasa = Amos (Gl, 1) 7 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


p wole = zeroak; Kp 


base prob = sum(n) / length (n); 
tol = le-8; % convergence criteria 


D 
I 


zeros(1, K); 

W = zeros(1, K); 

eit = EZ eroe TER) 
KOE 


1 


Neo" $ 
ae (D) = 0.005) 

x smth(1) - O0; 
120) (30). = Onil 
(a) es aa), 
Wi) = 0.0027 


b0 = log(base prob / (1 - base prob)); 


sus (Ue ee 1) 
x pred(k) - x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
esla) = se wyermüE(s = 3)5 
v_pred(k) = v_updt(k - 1) + ve(m); 
end 
x updt(k) = get posterior mode(x pred(k), v_pred(k), r(k) 
5 se) (du) o), vila), 190, uS) 5 
p updt(k) = 1 / (1 + exp((-1) + (bO + x updt (k)))) ; 
v dela) > iL / (Gl / y aldo) x (Gel (im) ~ 2) / sw (n) 
+ prupde (kh) (Gl = pmupdt (100 
end 
x smth(K) = x_updt(K) ; 
v_smth(K) = v_updt (K); 
W(K) = v smth(K) + (x smth(K) ^ 2); 
Ax: (end = 1)) Ev upat I (end! - 3) T vipredi(2zend)); 
mos las QUE = 35) 8 (630) eal 
x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - x pred(k + 


DF 
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v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 
v pred(k + 1)); 
CW(k) = A(k) + v smth(k + 1) + x smth(k) * x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
end 
if (m « M) 
R = get linear parameters(x smth, W, r); 
720) (my ce 3L) = IAL, IL) p 
ri(m + 1) = R(2, 1); 
vr(m + 1) = get maximum variance(r, rO(m + 1), rl(m + 1), 
W, x smth); 
ve(m + 1) = (sum(W(2:end)) + sum(W(1:(end - 1))) - 2 + 
sum(CW)) / K; 
mean dev = mean(abs([ve(m + 1) r0(m + 1) rl (m+ 1) vr(m + 
Tr ives) om) em) SO 
if mean_dev < tol 
fprintt (Aq cab) exei Wie Naka) Ex e. Malal «s Elche 
Wes Soe enu: c poca, tl, $E Sua (Eb), sel) (ud) seal (a), Awe 
(m), ve(m)); 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
precia (“ji ce SENO c E ao cs Eo cl E alike 
Nas o Solbes = Sole Naa, y) Se Ema (Eb), am) + al), seal (im 
4 35). ela. 1), sem > 1) 
eel cx mos (al, HQ 7 
v_pred = zeros(1, K); 
se Whole = Aros (a, 19) 7 
y viole cx meros, 19) 7 
amen (els emel) = memos (al, i = 35 $ x smth(1) 
needed for next iteration 
yv anda = aros (al, HQ E 
9) Wherche = meros (al, 197 
A = zeros(1, K); 
W = zeros(1, K); 
CW = zeros(1, K); 
(Ge renos (al, E) 7 
end 
end 
end 
|o Wicks = db c4 (Gl & Sel) sx (190 <b ox 109615) ) 5 
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p smth = 1 ./ (1 + exp((-1) * (b0 + x _smth))); 

r smth = r0(m) + rl(m) * x smth; 

n plot - NaN » ones(1, K); 

in Jede (ía > 0) = 17 

figure; 

subplot (511); 

hold on; 

piloter yn ko linewidth, v5) 

patehl(xp IT ODF O A255 GI color E none e 
FaceAlpha', 0.3); 

grid; 

ylabel('(a) z (k)'); title('State Estimation with Experimental 
Daca 

set(gca,'xticklabel', []); ylim([(min(y) - 1e2) (max(y) + 1e2)]); 

&ubpillotq 512). 

Siente, 3m jede, “exi, “colez”, JH 0,75 Ol); 

¡pelea (E3a,) yp t 1.25, Mas, 192, 1921) / 255, "isses. “mae”, 
'FaceAlpha', 0.3); 

ym ome 25106; 

ylabel('(b) n (k)'); 

grid; set(gca,'xticklabel', []); 

subplot (513); 

hold on; 

GS (ie, ae, "94g 

loe fe, de Sel, “2, “lime, do) p 

arca) Wo, (1192 192, 192% / 255, EdgeeoloDn "means y Y 
FaceAlpha’, 0.3); 

ylabel('(c) r_([k)'); 

grid, set (gcea,’xticklabel’, [m 

subplot (514); 

hold on; 

plo e, jo Ema, “Colen, RA / 255), ©, Goa / 255), * 
ala, LS AD) 

etc (Ej) Mo 0.5), Moa, 192, 19211 / 285, “mkeo; “messi, 4 
FaceAlpha’, 0.3); 

ylabel('(d) p (k)'); 

grid; set(gca,'xticklabel', []); 

subplot (515); 

hold on; 

[olo (f, se muda, "calor", Mos / 285), ©; (uus / 255); " 
Iie ward) 25) 

wL a Vals 

YPX = YP; 

ypx(yp == 0) = yl(1); 

ypx(yp == 1) = yl(2); 

paten (xp) yox, [1925 19275192] / 255, Edgecolor' *none’ , i 
FaceAlpha’, 0.3); 


ylabel('(e) x (k)'); 
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xlabel('time (min)'); 
grid; ylim([(min(x smth) - 2) inf]); 


function [y] - get posterior mode(x pred, v pred, z, r0, r1, 


bo, n) 
M - 100; % maximum iterations 
it = zeros(1, M); 


2 S zeros Gio) > 
df = zeros(1, M); 


p 
vt: 
m 
1 


x pred; 


for 1 s Lie] 433 


C = v pred / ((rl * 2) * v pred + vr) 


iE (35) = ae) = zs joel = € o (Qu ws (e 
+ vr x (n- (1/ (1 + exp((-1) » (bO + it 

df(i) = 1+ C * vr x exp(bO + it(i)) 
a) )) ~ 235 

abie (ak se dj) e alte (at) = E) / heal) y 

e close (ak <5 3D) = LET )) es lemi 


y m dell sb Lya 
return 
end 
end 


(i 
/ 


error ('Newton-Raphson failed to converge.’) ; 


end 
function y = get_maximum variance(z, 


K = length(x_smth) ; 


A 


x e (as wl ab ike (r0 
= 2 
r0 « rl * sum(x smth)) / K; 


* 
* 


end 


2) s fau ^ 2) 
250 sum (zo) NE es sal cols Ema, 


120), wily Wy 2 Sma) 


function y = get linear parameters(x smth, W, 


K - length(x smth); 


y = [K sum(x smth); sum(x smth) 
x smth)]; 


end 


sum (W) ] 


\ 


* sum(W) 


z) 


[sum(z) ; 


Sum L Z 
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WE, 


0 = ri «o x pred) 
DYDD 
(al 4 990) <= sel 


zZ) +2 + 


.* 
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10.3 State-space Model with One Binary and Two 
Continuous Observations 


10.3.1 Simulated Data Example (aI, Excluded) 


load('data one bin two cont no extern stim.mat'); 


base prob - sum(n) / length(n); 


$$ parameters 


M = 186; $ maximum iterations 

mM = ; 

tol = 1e-8; % convergence criteria 

DON SNTE EOS SIAM) E % binary GLM model 


dl c mermas ll, IM) y 


20 = temo, Wis % continuous model 

ri = zeros (1, M): 

vr = zeros(1, M); % continuous model noise variance (1) 
S0 = zeros(1, M); % continuous model 

El e sesso, Ws 

vs = zeros(1, M); $ continuous model noise variance (2) 
ve - zeros(1, M); $ process noise variance 

rho = zeros(1, M); % random walk forgetting factor 


K = length(n); 


pie predg-NExeros 19) 7 
VEDPredE-Nceros 9) 7 


x updt = zeros(1, K); 
tos = Amos ll, 10) 7 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


pEupdtE-acerosH EHE 


1 


D 
I 


zeros(1, K) 
= eros. x 
K 

K) 


= 
I 


Ya 


1 


CW = zeros(1, 
C= zeroes i, 


$$ initial guesses 


bo (1) 
ior ((3L) 


log (base prob / (1 - base prob)); 
1; 


Il 


Il 
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; $ guess it's the first value of r 
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as Me ee T) 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
da) > sado) S > ole (is dE 
v_pred(k) = (rho(m) * 2) * v_updt(k - 1) + ve(m); 
end 
C(k) = v pred(k) / (vr(m) « vs(m) + v pred(k) * ((r1(m) ^ 
2) * vs(m) + (s1(m) ^ 2) « vr(m))); 
upd t (ks) = gets posterilonmsmode Galpredi (ki) Cd) Ol 
m), rl(m), b0(m), bl(m), vr(m), n(k), s(k), sO(m), sl(m), vs( 
m)); 
p updt(k) = 1 / (1 + exp((-1) « (bO(m) + b1(m) « x updt (k 
2333) 8 
yc x ak / (GL ^ vado) = (El ^ 2) / vel) 
+ ((si(m) ^ 2) / vs(m)) + (b1(m) ^ 2) « p updt(k) « (1 - 
p updt (k))); 
end 
x smth(K) = x updt(K); 
v smth(K) - v updt(K); 
W(K) = v smth(K) + (x smth(K) ^ 2); 
AG (eei e 1)) cx seme) c wr tele (ls (enel = 3) M v pred(2:end 
E 
for k= Ud CL 
x smth(k) = x updt(k) + A(k) « (x smth(k + 1) - x pred(k + 
AER 
v smth(k) = v updt(k) + (A(k) ^ 2) « (v smth(k + 1) - 


v_pred(k + 1)); 


CW(k) = A(k) * v smth(k + 1) + x smth(k) * x smth(k + 1); 
(k) 


W(k) = v_smth(k) + (x smth(k) ^ 2); 


prev = [rO(m) r1(m) ve(m) vr(m) rho(m) sO(m) si(m) vs(m)]; 
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R = get linear parameters(x smth, W, r, K); 
S - get linear parameters(x smth, W, s, K); 


b0(m + 1) = log(base prob / (1 - base prob)); 

bl(m + 1) = 1; 

rho(m + 1) = sum(CW) / sum(W(1:end - 1)); 

ve(m + 1) = (sum(W(2:end)) + (rho(m + 1) ^ 2) * sum(W(1: (end 
- 1))) - 2 x rho(m + 1) + sum(CW)) / K; 

vot + D = RCL, 4) 

Sim + Ae Ri, ds 

eu = 3) eg Ws 

slum + 1) = S(2, d$ 

vr(m + 1) = get maximum variance(r, rO(m +1), rl(m + 1), W, 

x smth, K); 

vs(m + 1) = get maximum variance(s, sO(m + 1), sl(m+ 1), W, 

x_smth, K); 

next = [rO(m + 1) rl(m + 1) ve(m + 1) velm + 1) rho(m + 1) sd 


(m + 1) sl(m + 1) vs(m +1)]; 
mean dev = mean(abs(next - prev)); 


if mean dev « tol 

fprintf('Converged at m = %d\n\n’, m); 

break; 
else 

ijspesheueis (7m c EN c te lene Woke! = o iaa E dee wel els) 
= %.18f\nsl = %.18f\nvs = %.18f\n\nve = %.18f\nrho = %.18f\n 
NIME 
m e 1. com + 4), lim = 1), vrim + 1), sims 1), slim * 
1), veím + 1), ve(m + 1), rho(m + 1)); 


x pred = zeros(1, K); 
prede meros, 19) $ 


se goleo >= PEroa lll, 19) y 
v_updt = zeros(1, K); 


se ama eme) = cos (il, ke = al) p s x smth(1) needed 
for next iteration 
v_smth = zeros(1, K); 


p_updt = zeros(1, K); 


A = zeros(1, R); 
W = xeros(i1, k); 
K); 
K); 


T 


CW = zeros (1, 
C = zeros(1, 
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end 
end 


$$ calculate confidence limits 

p smth = 1 ./ (1 + exp((-1) « (bO(m) + b1(m) « x smth))); % mode 
, lower and upper confidence limits for binary distribution 

r smth = r0(m) + rl(m) » x smth; 

s smth = s0(m) + s1(m) * x smth; 

%% plot graphs 


figure; 


subplot (511); 


hold on; 

plUotilp b 

lao imei, 522,7, Clbsherwpueles , db) p. ese 

plot(find(n == 0) - 1, 1.4 « max(p) « ones(length(find(n == 0))), 
'ks', 'MarkerFaceColor', 'k', 'MarkerSize', 4); 

plot (find(n == 1) - 1, 1.4 « max(p) + ones(length(find(n == 1))), 
'gs', 'MarkerFaceColor', 'g', 'MarkerSize', 4); 


ylabel('(a) p (k)'); ylim([o 0.181); 
title('State Estimation with Simulated Data'); 


subplot (512); 


hold on; 
lora, dee 
plotr smeh, 2,7, "leida MET 2) grid; 


ylabel('(b) r [k)'); 


subplot (513); 


hold onm; 
pilots o mb) 
jolies (a ceuta, seco. "dbsasuhssieln/ n dios) Crale 


ylabel('(c) s_{k}'); 


subplot (514); 


holdi on; 
akese ea SR 
ploti Emela, Uo, TTT 


ylabel('(d) x (k)'); xlabel('time index’) ; 


subplot (515); 

qqplot (x - x smth); 

title('00 Plot - State Estimate’, 'FontWeight', 'Normal'); 
ylabel('(e) input quantiles'); 

xlabel('standard normal quantiles'); 

grid; 


$$ supplementary functions 


Function EcEgotposteriommodexoipi ec (E), 12, 350, Ll, 190, Joi, wae, 
ia, E) SO, El, WS) 
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M = 200; % maximum iterations 
it = zeros(1, M); 
i8 c3 Mero, us 
df - zeros(1, M); 
she (a) = sm gere 
one al ex abe (ii = 3b) 

Ea) = del(a) = rs pree C s (sal se we (E = x9 = weil ow 
Xx pred) + sl « vr * (s - sO - sl * x pred) + 

vr + VS x bl x (n - (1 / (1 + exp((-1) * (bO + bl + 

BLE (2) 9 DIO 

df(i) =1 + C « vr « vs x (bl ^ 2) « exp(b0 + b1 « it(i)) 
J (GL s elo = eu ce dec) ^ 2) 

abeat se 3D) e mel) = EG) / ele (ab) n 

alse alos (el 3) = Dem 


y lie (al 2 
return; 
end 


end 


error ('Newton-Raphson failed to converge.'); 


end 
function y = get maximum variance(z, r0, r1, W, x smth, K) 
y e (eu se 9^2) s Gl > 2) e mu) 
= Lows xe) c» suin(4) = Bw wil c cor (fs Suda, E) d Bs 
r0 « rl «x sum(x smth)) / K; 
end 
function y - get linear parameters(x smth, W, z, K) 
y = [K sum(x smth); sum(x smth) sum(W)] \ [sum(z); sum(z .x* 
x smth)]; 


end 


10.3.2 Simulated Data Example 


load('data one bin two cont.mat') 


base prob sum(n) / length(n); 


$$ parameters 


M 


1e6; % maximum iterations 


1 


10.3 State-space Model with One Binary and Two Continuous Observations 133 


mE 
tol = 1e-8; % convergence criteria 
bO = xerosil, M); $ binary GLM model 


bl = zeros(1, M); 


roO = xeros(i, Mi); % continuous model 
rl - zeros(1, M); 
yr oz teros (1, My % continuous model noise variance (1) 


continuous model 


oe 


S0 = zeros(l, M 
sl = zeros(1, M); 


vs = zeros(1, M); % continuous model noise variance (2) 
ve - zeros(1, M); $ process noise variance 

rho = zeros(1, M); % random walk forgetting factor 
alpha = zeros(1, M); $ external input gain parameter 


K = length(n); 


x pred = zeros(1, K); 
v pred - zeros(1, K); 


Pe ose c zaco, 19) E 
VAULPAEA= ZST SNE 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


p ugli = wre (Gal, EHE 


1 


D 
I 


zeros(1, K) 
zerom(t, K) 
K 

K) 


= 
I 


Ye 


Z 


CW = zeros(1, 
Qo = zeros 1, 


$$ initial guesses 
= log(base prob / (1 - base prob)); 


o 


; $ guess it’s the first value of r 


$$ main function 


tor m=, 1M 
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if (k == 1) 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
x pred(k) = rho(m) » x updt(k - 1) + alpha(m) + I(k); 
v pred(k) = (rho(m) ^ 2) * v updt(k - 1) + ve(m); 
end 
C(k) = v pred(k) / (vr(m) » vs(m) + v pred(k) * ((r1(m) i 


^ 


2) x vs(m) + (s1(m) 2) * vr(m))); 
x Updt i(k) = get posteriion modeiGapredi(k) eee 
m), ri(m), bO(m), bi(m), vr(m), n(k), s(k), so(m), si(m), vs( 


p updt(k) = 1 / (1 + exp((-1) +» (bO(m) + b1(m) « x updt (k 
DEDE 
y wei cx 3b ^ (GL X voca) = (El ^ 29) 7 vel) 
+ ((s1(m) ^2) / vs(m)) + (b1(m) ^ 2) » p updt(k) « (1 - 
p updt (k))); 
end 
x smth(K) - x updt(K); 
v_smth(K) = v updt(K) ; 
W(K) = v smth(K) + (x smth(K) ^ 2); 
A Eend n Saa) y Yi tgele (ala (sal > KI 27. v pred(2:end 
Vi 
ere He ex (ue > id) (3b) gal 
x smth(k) = x updt(k) + A(k) + (x smth(k + 1) - x pred(k + 
E 
v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 


v pred(k + 1)); 


CW(k) = A(k) « v smth(k + 1) + x smth(k) « x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
end 
prev = [alpha(m) rO(m) r1(m) ve(m) YY (m) rho(m) sO(m) sal (m) 
vs (m)]; 
R - get linear parameters(x smth, W, r, K); 


S - get linear parameters(x smth, W, s, K); 


(9 e lermes = 30)) (a (Agciceh) i» TO (yel = 3) 795 
(I(2:end) « x smth(1:(end - 1))') (I * I')] \ [sum(CW); ( 
mo: end) oe mea (2 enc) i; 


b0(m + 1) = log(base prob 2 (ah = base prob) )j; 
bl(m + 1) = 1; 
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alpha(m + 1) = Q(2, 1); 
ve(m + 1) = (sum(W(2:end)) + (rho(m + 1) ^ 2) « sum(W(1: (end 
- 1))) - 2 x rho(m + 1) * sum(CW) - 

2 * alpha(m + 1) « (I(2:end) * x smth(2:end)') + 2 « 
alpha(m + 1) + rho(m + 1) « (I(2:end) « x_smth(1: (end - 1))") 
ania eds 

SR SIRSE > 10) > 3) x (abo: x79) Y gg 
vod + y = RCL, do 
weil (im s» 3D) = i2(2, ds 
sota 2 3) = Su Us 
Sil x 1) = S, ll 
vr(m + 1) = get maximum variance(r, rO(m + 1), rl(m + 1), W, 
x smth, K); 
vs(m + 1) = get maximum variance(s, sO(m + 1), sl(m+ 1), W, 
es mila, 9E 
next = [alpha(m + 1) rO(m + 1) rl(m + 1) ve(m + 1) vr(m + 1) 


rho(m + 1) sO(m +1) s1(m + 1) vs(m +1)]; 
mean dev - mean(abs(next - prev)); 


if mean dev « tol 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
pri (70 sd\nrO = %.18f\nrl = %.18f\nvr = %.18f\n\ns0 
SAME = E Ea = Condes ae = a EN aano e Eo EN 
naljpha = s. Us—i\n\n", 
m 4 1, rO(m + 1), riim + 1), velm + 1), sOim + 1), slim + 
1), vs(m + 1), ve(m +1), rho(m + 1), alpha(m + 1)); 


e Il 


sore = meros, iE 
vo) > peros lll, 19 y 


x_updt = zeros(1, K); 
y Woste. c Heros, 19) $ 


se miele rend) c vcueei(al, ke = al) p s x smth(1) needed 
for next iteration 
v smth = zeros(1, K); 


jo ase = rog, 1) p 


A - zeros(1, K); 
W = xeros(l, Ki; 
K) ; 
KIE 


T 


CW = Zeros (i, 
poc Heros, 
end 
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%% calculate confidence limits 


p smth = 1 ./ (1 + exp((-1) x (bO(m) + b1(m) « x smth))); % mode 
, lower and upper confidence limits for binary distribution 

r smth = r0(m) + rl(m) * x smth; 

s smth = s0(m) + s1(m) + x smth; 


$$ plot graphs 


figure; 

subplot (511); 

hold ou; 

pilot! pb) 

[olo (o smda, Sr- na "übheeihehem" y A 25) Eie 

plot (find (n == 0) - 1, 1.2 * max(p_smth) « ones (length(find (n == 
0))), ‘ks’, 'MarkerFaceColor', 'k', 'MarkerSize', 4); 

olor Meaca a uM a ea a e Elenea e 
1))), ‘gs’, 'MarkerFaceColor', 'g', 'MarkerSize', 4); 


ylabel('(a) p (k)'); ylim([o 0.171); 
title('State Estimation with Simulated Data'); 


súbplot (512); 


holdon; 
piotr deu e 
plow (6 Emilia, "5.7, H ERIE REI  di2859 s gnsielg 


ylabel('(b) r [k])'); 


subplot (513); 


hold onm; 
pietre) 
les (e sua, "c.t. iaae a al AS) alel 


ylabel('(c) s_{k}'); 


subplot (514); 


hold on; 

aires, 19%) 

plot smeh, e linewidth 25) grid; 

¡lo tetas 0) = ak, (6) Xonesiengtehtxmd!te——9)9 0 S ss Mur 
MarkerFaceColor', 'k', 'MarkerSize', 4); 

plot tnde —— ASS Eonestensgthittimdim-—9 vg cs T 
MarkerFaceColor', 'c', 'MarkerSize', 4); 


ylabel('(d) x (k)'); xlabel('time index'); 


subplot (515); 

qqplot (x - x smth); 

title('00 Plot - State Estimate’, 'FontWeight', 'Normal'); 
ylabel('(e) input quantiles'); 

xlabel('standard normal quantiles'); 

grid; 


$$ supplementary functions 


Function EcEgotposteriommodexoipi ec (E), 12, 350, Ll, 190, Joi, wae, 
ia, E) SO, El, WS) 


10.3 State-space Model with One Binary and Two Continuous Observations 137 
M = 200; % maximum iterations 
aie ex aro Ll, IW) E 
i8 c3 Mero, us 
df - zeros(1, M); 
she (a) = sm gere 
one al ex abe (ii = 3b) 
Ea) = del(a) = rs pree C s (sal se we (E = x9 = weil ow 
Xx pred) + sl « vr * (s - sO - sl * x pred) + 
vr + VS x bl x (n - (1 / (1 + exp((-1) * (bO + bl + 
ae (05) VN) B 
df(i) = 1+ C « vr « vs x (bl ^ 2) « exp(b0 + b1 « it(i)) 
J (GL s elo = eu ce dec) ^ 2) 
abeat se 3D) e mel) = EG) / ele (ab) n 
alse alos (el 3) = Dem 
wee duela 4 AL) ¢ 
return; 
end 
end 
error ('Newton-Raphson failed to converge.'); 
end 
function y = get maximum variance(z, r0, r1, W, x smth, K) 
y e (eu se 9^2) s Gl > 2) e mu) 
= Lows xe) c» suin(4) = Bw wil c cor (fs Suda, E) d Bs 
r0 « rl «x sum(x smth)) / K; 
end 
function y - get linear parameters(x smth, W, z, K) 
y = [K sum(x smth); sum(x smth) sum(W)] \ [sum(z); sum(z .x* 
x smth)]; 


end 


10.3.3 Experimental Data Example 


load('expm data one bin two cont no 


(«Iņ Excluded) 


extern stim.mat'); 


min scr thresh 
min scr prom 


BE = Ap 


(0), 05, 
min scr thresh; 


€ 


(0: (length (phasic) - 1)) / fs; 
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ph phasic; 
tn tonic; 
Pe (onesie; = VI. 


[pks, locs] = findpeaks (ph, 'MinPeakHeight', min scr thresh, ' 
MinPeakProminence', min scr prom); 


r = interpl([1 locs length(ph)], log([ph(1) pks ph(end)]), 1: 
length(ph), 'cubic'); 
Gi ex icialp 


ni zeros (ye enc). 
I = zeros(1, length(r)); 
milocs) = ly 


base prob = sum(n) / length (n); 


std_s = std(s); 
Stel se Steel (Ge) s 


Bee At 
mee f heel ep 


$$ parameters 


M = 5e5; % maximum iterations 
tol = 1e-8; % convergence criteria 


bO = zeros(1, M); $ binary GLM model 


ro = zerosil, M); % continuous model 

rl = xeros(l, M); 

vr s zeros(1, M); % continuous model noise variance (1) 
SO = xerosii. M); % continuous model 


ve = zeros(1, M); % continuous model noise variance (2) 
ve = zeros(1, M); $ process noise variance 
rho = zeros(1, M); % random walk forgetting factor 


K = length(n) ; 


pe mel = emos (aL, 0) 7 
Mz joramel = emos (al, 19) E 


Pe ilerele = SOS 
Ko acle = zaros, 19) 7 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


plupart- Zeros MIRE 
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A - zeros(1, K); 
W - zeros(1, K); 
CW - zeros(1, K); 
(G cx venga (i, Tp 
$$ initial guesses 
b0(1) = log(base prob / (1 - base prob)); 
DIS ilg 
r0(1) = r(1); % guess it's the first value of r 
r1(1) = 1; 
s0(1) = s(1); 
aray alee 
vet) = 0.05; 
Veil) = 00055 
ve(1) = 0.05; 
rode cde 
$$ main function 
for m = 1:M 
Lor ie = Lais 
if (k == 1) 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
x pred(k) = rho(m) * x updt(k = 1); 
v_pred(k) = (rho(m) * 2) * v_updt(k - 1) + ve(m); 
end 
C(k) = v pred(k) / (vr(m) * vs(m) + v pred(k) * ((r1(m) R 
2) « vs(m) + (sl(m) ^ 2) « vr(m))); 
x updt(k) = get posterior mode(x pred(k), C(k), r(k), r0( 
m), rl(m), bO(m), bl(m), vr(m), n(k), s(k), sO(m), sl(m), vs( 
m)); 
p updt(k) = 1 / (1 + exp((-1) « (bO(m) + b1(m) « x updt (k 
DEDE, 
v updt(k) = 1 / ((1 / v pred(k)) + ((r1(m) ^ 2) / vr(m)) 
+ ((s1(m) * 2) / vs(m)) + (b1(m) ^ 2) x p updt(k) « (1 - 
p updt (k))); 
end 
x smth(K) = x updt(K); 
v smth(K) - v updt(K); 
W(K) = v smth(K) + (x smth(K) ^ 2); 
Mo (exei e 1)) = seme) c» Y wysele(Lg (eme! = 31) ajk v_pred(2:end 


140 10 MATLAB Code Examples 


x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - x pred(k + 
1)); 
v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 
v pred(k + 1)); 


CW(k) = A(k) * v smth(k + 1) + x smth(k) + x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
if (m « M) 


R - get linear parameters(x smth, W, r, K); 
S - get linear parameters(x smth, W, s, K); 


b0(m + 1) = log(base prob / (1 - base prob)); 

bi(m + 1) = 1; 

rho(m + 1) = sum(CW) / sum(W(1:end - 1)); 

ve(m + 1) = (sum(W(2:end)) + (rho(m + 1) ^ 2) * sum(W(1:( 
end - 1))) - 2 x rho(m + 1) + sum(CW)) / K; 


if (abs(get_maximum_variance(r, R(1, 1), R(2, 1), W, 


Ema, 1) = Cite maximumivarience (a, SL, a), SIA, a, We, 
2s amelan KOD a oul) % overfitting check 

rom + T) e rom) 

ri(m + 1) = r1(m); 

som + 1) = sO(m); 

ial Git a 3D) = SL (im) y 

vr(m + 1) = vr(m); 

vs(m + 1) = vs(m); 

mean dev = mean(abs([ve(m + 1) rho(m + 1)] - [ve(m) 
rho(m)])); 

else 

20 (ad + 3D) x RE, al) 5 

ium e 3) e m. 15 

SO Ga e i) esp E); 

slum + 1) = Sie, X 

vr(m + 1) = get maximum variance(r, rO(m + 1), r1(m + 

Wa Wy sx Elia, 10) y 
vs(m + 1) = get maximum variance(s, sO(m + 1), s1(m + 


mean dev = mean(abs([rO(m + 1) r1(m + 1) ve(m + 1) vr 
(moe 1) -xho(im - 1) s0(m - 1) stum 1) veía + D] = 2... 
[rO(m) ri(m) ve(m) vr(m) rho(m) sO(m) si(m) vs(m) 
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if mean_dev < tol 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
ope (Uy > SENO cx Eo debe all = Ea leia in Woke) e 
\nrl = %.18f\nvr = %.18f\n\nsO = %.18f\ns1 = %.18f\nvs = %.18 
f\n\nve = %.18f\nrho = %.18f\n\n’', 
me 1 bolm + D biim + L), ron x 3), cla + 1), we 
(m + 1), som + 1), slim - 1), vela + 1), veim + 1), rho(m + 
2» 5 
x pred = zeros(1, K); 
vee cs aos (il, 19) 7 
ase e moro al, MR) 
y tgele = zeros (al, 19) 7 
se Ema acus) c eno, E = di), $ x smth(1) 
needed for next iteration 
v smth - zeros(1, K); 
B Worle cx EET HE 
A - zeros(1, K); 
W = zeros(i1, K); 
CW - zeros(1, K); 
(G e rere. E) 
end 
end 
end 
$$ calculate confidence limits 
p smth = 1 ./ (1 + exp((-1) x (bO(m) + b1(m) « x smth))); % mode 
, lower and upper confidence limits for binary distribution 
r smth = exp(std r * (rO(m) + r1(m) * x smth)); 
s smth = (sO(m) + s1(m) + x smth) * std s; 
els no nin vi 025). xusmthy sas (vesmth))) 
uci = "norminv,(0).975,, xwsmth; sqetvssmth))r 
lei) = mom al, op 
eich, J) ex reza, 1) y 
for koc ask 
[1cl p(k), ucl p(k)] = get pk conf lims(v smth(k), b0(m), 
x smth(k)); 
end 
certainty = T - normcdt (prcetile G smth, 50) x ones(1, Xength( 
x smth)), x smth, sqrt (v_smth) ); 


$$ plot graphs 
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disp plocting n. E 


xp_fs_plot = 4; 


index = (0:(K - 1)); 
t_index = index / fs; 
re darles = (US = 30 2((-30) 80) / ie, 


transp- Ors; 


subplot (611); 


hold on; 

pllotteimdex xiorig n kR test hb 

loe (faire c 0) Y ES, S27 «* ames Ulenacjaa (Calaci(a cx 0))), CSS. 7 
MarkerFaceColor', 'k', 'MarkerSize', 5); 

pior tindiim es 30) / Es, Do ornes Mengt Mie inani == 1))), “ee, * 
MarkerFaceColor’, 'g', 'MarkerSize', 5); 

ylim([O 4]); yl = ylim; 

postccla (bso), (2), z$9(2)), esa) Y 299 Es iex; [yal (a) wala) wl 
(PIE acen On2).  Edgecolon none”); 

osea (sa), S493), o). oa) 4 san Es aloe, (yal) yall) yl 
(2) yuan “gt” PEacealpha”, 0.2, “EdgeCollor’, none”); 

patch([xp(3), xp(4), xp(4), xp(3)] / xp fs plot, [yl(1) y1(1) yl 
(2) y, 10,670 Ol,  paceniohad , 0,2, eee, 
none'); 

patch([xp(4), xp(5), xp(5), xp(4)] / xp fs plot, [yl(1) y1(1) yl 
(2) y RI paceAlpha^ 052) “EdgeCollor’, 'mone'), 

Sres), O), S305), 395) 1174 sac: ses) oe, lA) sre (b) yal 
(PIE ty shaceAlpha’ 0.2, ORdgeGoillor ^ nonen), 

ylabel([' (a) skin cond.', ' (mu S)']); grid; xlim([O (xp(6) / 
eo ES la) 11) 5 

set (gca, 'xticklabel', []); 


title('State Estimation with Experimental Data’); 


subplot (612) ; 


hold on; 

plot (e aces, emt croi Mr 0.3 0, “tasas, alo) — 

plo (dE soles, exalta <> siecle), "else"; 10 0,9 Wl), “limita, 
1.51) 

grid; 


ano (Scpi( 6) A XD ESp:O t) 
ylim([(min([exp(r « std r) r smth]) - 0.25) (0.25 + max([exp(r « 
mel x2) ae Emil) 1) wal e walang 


parca (bs), (a), 39012), e391) Y 239 xm joiloe, NL) db) sal 


(PISA acen m O02, E dgetollor none) 
seca (Esa), a. a). aaa) 4 san Ee loc, li) YA) s 
(2) y2); gu “haceAlpha’, 0.2, Edgelolor”, “none”); 
patch([xp(3), xp(4), xp(4), xp(3)] / xp fs plot, [yl(1) y1(1) yl 

(PISA AOS H STE 0), Eacexlphac 02, Bdgecolor'^, 


none’); 
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patch([xp(4), xp(5), xp(5), xp(4)] / xp_fs_plot, [yl(1) yl(1) yl 
(2) yan @iil Yat). tmacenioha 0f2,) tEdgeColon o “none” ); 
¡rca MS), O), s395)1] 7 29 ies loc, yb) sab (b) sl 
(Ey yankee ys ac enla 02 “BdgeCollor’ = “none” ) i: 

set (gca, 'xticklabel', []); 

ylabel (' (bB) phasic’); 

subplot (613); 

held on; 

alore mees, e Smena, "x^, “eslez”, [0,5 (25 955) (66 / 255) 1, 
‘linewrdth”, 1.5); 

lore (te abaci», E c» Eee sy “soles”, (1 0.5 (ue / B55), linewidth 
^g Io BR epele 

xlim([O (xp(6) / xp fs plot)]); 

ylim([(min([(s * std s) s smth]) - 0.25) (0.25 + max([(s * std s) 
cx maze] py wal c valio 

pacchi kas (i); 939020) 5 r39(29)5 esol) 4 rss ES loc, lia) AL) sal 
(2) yi) “xt bWacenipha' 0.2, “EdgeColor”, 'mone”), 

pecca kaala) sse)» S33), sso02)1 7 239 ses joiloe, [hy (ab) yal (al) sal 
(QQ vans “gi, Bacenipha^ 0:2, “Edgecollor’, nonen), 

pacek (S); Saal), ss9(2))5 sae] y s tes) lore, bab) sal (a) sal 
(2) y1(2)], [1 0.647059 0], 'FaceAlpha', 0.2, 'EdgeColor', 
none’); 

patch([xp(4), xp(5), xp(5), xp(4)] / xp fs plot, [yl(1) y1(1) yl 
(2) ye), “bt, “Eaceñlpha”, 01.2, “EdgeCollor’, 'none^); 

¡arca Es), 39169) 5» 310), sso(5)1 y so tes) lore, bab) sal (al) yal 
(2) dL) uu 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

set(gca,'xticklabel', []); 

ylabel('(c) tonic'); 

subplot(614); 

hola onm; 

alore sele, + mea, color MEE mew M5) 9 Spei 

anb (lhe T xe alles] y eub +s milo (el zs] “e, "gere" y 

'none', 'FaceAlpha', 0.5); 

yam E (oce emt h) = 0425) (025 => merce Ema) li Val = Falo 

¡arca oa), (2), sso(2), Bou) Y Seo Es alor, [ub sil (a) y 
(PISA acera 02  EdgeCollor’) none”); 

parch (Esa), a. o). 22) 7 ean Es aaloe, laa) (b s 
(2) yi), gl, “RaceAlpha’, 0.2, “EdgeCollor’, 'none^); 

patch([xp(3), xp(4), xp(4), xp(3)] / xp fs plot, [y1(1) y1(1) yl 
EN) SAG, ML 0,4705 0, pace pado ccecolozc 
none'); 

patch([xp(4), xp(5), xp(5), xp(4)] / xp fs plot, [y1(1) y1(1) yl 
(2) yea “be Racenlpha 0:2, “EdgeCollor”, "none, 

arcas), E), S305), ss9(5)1 7 239 ses oe, la) FAL) sil 
(2) AB, oye" 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 
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xlim([O (xp(6) / xp fs plot)]); 


set (gca,’xticklabel’, []); 
ylabel('(d) state (x_{k})’); 


subplot (615); 
Pie 


ITE Emela, “50, T T T y Gheatcls 

TTT e T G TT deu T E T jo) Il, (lal, (Oy (127 / 
255)], 'EdgeColor', ‘none’, 'FaceAlpha', 0.3); 

xlim([O (xp(6) / xp £s plot)]); 

ylim([0 (max(p smth) * 1.5)]); yl = ylim; 

arcas a), (2), 08), ss) Y 239 es eee, lhyl (a) 2 (al) sl 
(2) ydo) a cea 0225) E dgetollor nonet) 

jocueelafleso(2)), sa), oa), sso(2)] y B49 Te lore, ba) sal (aly) yal 
(2) y2) tg", PEacenipha”, 0.2, “EdgeCollor’, 'none^)r 

patch([xp(3), xp(4), xp(4), xp(3)] / xp fs plot, [yl(1) yl(1) yl 
(My 2 MICI O0, 647058 0, Eacezpha 0,2, Bdcgecolozr 
none'); 

jouera (qo), Saal), so), sao) y so Te lor, (bab) sal (aly sal 
(A) yankee. {bt macenioha 02 5 sBdgeCollor >. none”) 7; 

patch([xp(5), xp(6), xp(6), xp(5)] / so ree) jolie, (fyb (al) sab (al) aval 
(2) y2) “yo. Raceniphia s 0.2, “EdgeColor”, none”) 

set (gca,’xticklabel’,[]); 

ylabel(('(e) probability’, p (k))'], 'FontSize', 11); 

subplot (616); 

Bold our 

vl = [0 0.9. Elena) 0.9; Efend) 1; 0 t]; 

cl = [1 (220 / 255) (220 / 255); 1 (220 / 255) (220 / 255); 10 
Op 3E (9) Ol) 

facesl = [Ml 2 3 4]; 

patch('Faces', facesl, 'Vertices', vl, 'FaceVertexCData', cl, ' 
FaceColor', 'interp', A 
'EdgeColor', ‘none’, 'FaceAlpha', 0.7); 

v2 = TO 0; eene 0s tend) 0.1; 0 051]5 

G2 = [© 0,8 07 0 0.8 Of (204 / 255) 1 (204 7 285), (204 / 255) 1 
(Ent 7 255) 5 

faces? = [1 2 3 4]; 

patch('Faces', faces2, 'Vertices', v2, 'FaceVertexCData', c2, ' 
FaceColor', 'interp', ; 
'EdgeColor', 'none', 'FaceAlpha', 0.7); 

loa, csrralmez, "crees", Misa / 255) (9 / 255) (226 / S55); 

2 diesel , disi) p gridi 
xlim([O (xp(6) / xp £s plot)]); 


xlabel (’ 


(gym 


time 
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ylabel('(f) HAI’); 
$$ supplementary functions 


funcione = Siete jue: mee (Es reel, (Cp i, Op sail, 190, 1D, wae, 
ie, co ol, Mel 


M = 200; % maximum iterations 
alij > vacuae abo du 


f = zeros(1, M); 
dr a zeros M). 


it(1) - x pred; 
for 4 s Lim = 1) 
ie (ah) e alice (at) = x pued e Q o (Gell o wee (Qe x0 xL os 
x pred) + sl x vr « (s - sO - sl « x pred) + 
vr * VS x bl x (n- (1 / (1 + exp((-1) * (bO + bl x 
alte (aL) VOD DY) f 
df (i) = 1 + C « vr » vs x (b1 ^ 2) « exp(b0 + bl « it(i)) 
¿UL & eso + Toil oc ae) ^ 295 
ehm (ak sm 3D) e sequ) = az (al) / che) 
if abs(it(i + 1) - it(i)) « 1e-14 
wr = abe (al 4b 35) 5 
return; 
end 


end 
error('Newton-Raphson failed to converge.'); 
end 


iunctione Ce S ger als Com iinei Oh mss) 


p = (1e-4:1e-4:1); 
fp = cumtrapz(p, 1 ./ (sqrt(2 » pi » v) * p .* (1 - p)) .« 
Stud y (2 e sees (leslie of (GL = T9) e ckso(900))) = 2 
s a)y 


il ex heia Es 0.3975) y 
m e sexes. € 0.025) y 


ucl = p(n(end)); 


lcl = p(m(end)); 
end 


function y = get maximum variance(z, r0, rl, W, x smth, K) 
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= 2 o x) 2 enum) = 2 2 mil o clesie emen, va) om $ 
TO sumxesmth) SR; 


end 
function y - get linear parameters(x smth, W, z, K) 


y = [K sum(x smth); sum(x smth) sum(W)] MX [sum(z); sum(z .x 
x_smth)]; 


end 


10.3.4 Experimental Data Example 


load('expm data one bin two cont.mat'); 


min scr thresh - 0.015; 
min scr prom - min scr thresh; 
fs = 4; 

epoch =" 10; 


Sisley) =. il; 


stim = s data.aug stim; 
ph = s data.ph; 
tn - s data.tn; 


[pks, locs] = findpeaks(ph, 'MinPeakHeight', min scr thresh, ' 
MinPeakProminence', min scr prom); 

r = interp1([1 locs length(ph)], log([ph(find(ph > 0, 1)) pks ph( 
excrementa 


(sj = ages 
zeros(1, length(r)); 
I = zeros(1, length(r)); 


E] 
I 


n(locs) 
I (stim) 


1; 
T; 


ll 


std_s = std(s); 
Esel ss = iil (se) p 


Sse / auil ep 
me ae f ore ip 


$$ parameters 


M = 5e5; % maximum iterations 

tol = 1e-8; % convergence criteria 

fo) =S eros (PEME $ binary GLM model 
jou = memos (aL, Me 
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rO = xeros(i, M); % continuous model 

rel, c» mero, Ag 

vr = Heros (1, Mi; % continuous model noise variance (1) 
s0 = zeros(1, M); % continuous model 

Sal c euenit dg 

vs = zeros(1, M); $ continuous model noise variance (2) 
ve - zeros(1, M); $ process noise variance 

rho = zeros(1, M); % random walk forgetting factor 
alpha = zeros(1, M); $ input gain parameter 


K = length(n) ; 


<pred) = zeros (Gl, Kk); 
v pred = zeros(1, K); 


x updt - zeros(1, K); 
W wace = zaros kli, I7 


x smth = zeros(1, K); 
HZ Emela e Emo e 1) y 


[o Wiehe = am ll, 19) y 
jo. anda = zeroak; Op 


1 


D 
I 


zeros(1, K) 
zeros(1, K) 
K 

K) 


= 
I 


Ya 


L 


CW = zeros(l, 
C= eros, 


$$ initial guesses 


base prob = sum(n) / length(n); 


b0(1) = log(base prob / (1 - base prob)); 
Dis E 

roD- TEDE e guess its theif ir rst value Orn 
real (GL = ils 

SOG) e eua s 

grosor 

hse (Gab) = 0.037 

Wed 1000055 

veld) 04:055 

relay) (al) = abs 

alpha DESS; 


%% main function 
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x pred (ke) = G mtb)? 
v_pred(k) = ve(m) + ve(m); 
else 
x pred(k) = rho(m) » x updt(k - 1) + alpha(m) + I(k); 
v pred(k) = (rho(m) ^ 2) * v updt(k - 1) + ve(m); 


C(k) = v pred(k) / (vr(m) * vs(m) + v pred(k) < ((xi(m) ^ 
2) « vs(m) + (si(m) ^ 2) « vr(m))); 

x updt(k) = get posterior mode(x pred(k), C(k), r(k), r0( 
m), rl(m), b0(m), bl(m), vr(m), n(k), s(k), sO(m), sl(m), vs( 
m)); 

p updt(k) = 1 / (1 + exp((-1) « (bO(m) + b1(m) « x updt (k 
DODE 

y ce) = al / (GL ^ vrai) ex (Elm ^ 2) 7 ose } 
+ ((si(m) ^ 2) / vs(m)) + (b1(m) ^ 2) « p updt(k) « (1 - 
p updt (k))); 
end 
x smth(K) = x updt(K); 
v_smth(K) = v updt(K); 
W(K) = v smth(K) + (x smth(K) ^ 2); 
to eno! e 1)) cx semen) cs Y tele (ls (enel = 35) aff v_pred(2:end 
E 
for k= UE III 

x smth(k) = x updt(k) + A(K) * (x smth(k + 1) - x pred(k + 

IDDE 

v_smth(k) = v updt(k) + (A(k) ^ 2) x (v_smth(k + 1) - 
v_pred(k + 1)); 

CW(k) = A(k) * v smth(k + 1) + x smth(k) + x smth(k + 1); 

W(k) = v smth(k) + (x smth(k) ^ 2); 
end 


if (m « M) 


R - get linear parameters(x smth, W, r, K); 
S - get linear parameters(x smth, W, s, K); 


@ = [sum(wW(i:engd - 1)) (T(2:end) * x smth (ii: (end - 1))%), 


(scary se HET (sal e 3599 7) (ae es 307) \ [sum (CW 
)g (Qu9sewe x» se mim peus)" yd 7 


b0(m + 1) = log(base prob / (1 - base prob)); 
bl(m + 1) = 1; 
cota + dz Ol, Y 
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else 
alpha(m + 1) = Q(2, 1); 
end 
ve(m + 1) = (sum(W(2:end)) + (rho(m + 1) ^ 2) « sum(W(1: ( 
end - 1))) - 2 « rho(m + 1) x sum(CW) - 
2 * alpha(m + 1) + (I(2:end) * x smth(2:end)') + 2 «* 
alpha(m + 1) « rho(m + 1) + (I(2:end) + x smth(1:(end - 1))') 
+ 
(alpha(m + 1) ^ 2) * (I * I')) / K; 
if (absigetemassimumsverdiancel r Ri RS O T 
smc O = Getamaxamumevarlancel(sy SISTI, 
es finiri, 10) => oul) $ overfitting check 
rO(m + 1) = r0(m); 
clin + 1) = elim: 
sO(m + 1) = sO(m); 
sl(m + 1) = s1(m); 
wae x 3D) = was) 
ve(m + 1) = vs(m); 
mean dev = mean(abs([ve(m + 1) rho(m + 1) alpha(m + 
1)] - [ve(m) rho(m) alpha (m)]1)); 
else 
r0(m + 1) = R(1, 1); 
al (um <> 31) e d de 
Eu x 1) = S 3) 
sl(m +1) = S(2, 1); 
vr(m + 1) = get_maximum_variance(r, rO(m + 1), r1(m + 
I) gp Wy + min, 19) y 
vs(m + 1) = get maximum variance(s, sO(m + 1), s1(m + 


mean dev = mean(abs([rO(m + 1) rl(m + 1) ve(m + 1) vr 
(m + 1) rho(m + 1) alpha(m + 1) sO(m + 1) sl(m + 1) vs(m + 1) 


[rO(m) r1(m) ve(m) vr(m) rho(m) alpha(m) sO(m) sl 
(m vs(m)])); 
end 


if mean dev « tol 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
tpm (iu = CAAO a E PS mbi Seam m moo Ia 
\nri = HL = d8tNuums0 et lStNusdqd s 1er\nve = 3.18 
£\n\nve =o. 18f\nrho = s.1st\nalpha = < 1s8t\n\ne |, 
m + 1. boum + 1). biQu = 1). rom + 1), rim + 1), ve 
(m x 1), sO(m + 1), Slim + 1), vs m + 1), vem - 1), rhoim + 
1), alpha(m + 1)); 
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x pred = zeros(1, K); 
v pred = zeros(1, K); 
se Whole = os (a, 19) 7 
xr uste cx maroni, 19) 7 
es fem end) c menos al, M6 = 3$ $ x smth(1) 
needed for next iteration 
v anda eros MEE 
DEUpdUtN- moron (al, 197 
2 amda > os (al, 19) 7 
As zeros(1, Kj; 
W = zeros(1, K); 
CW = zeros(1, K); 
C c vaessexdh. 1) 7 
end 
end 
end 


$$ calculate confidence limits 


fp mode = 1 ./ (1 + exp((-1) + (bO(m) + bi(m) » x smth))); % 
mode, lower and upper confidence limits for binary 
distribution 

dell) cs aros (1, 197 

tel deo. = zroad; 0) 7 

r smth = exp((r0(m) + x1(m) * x smth) « std r); 

s smth = (sO(m) + s1(m) » x smth) * std s; 

Skn avg - get trial averages(s data, x smth, epoch, fs, 'skn'); 

X avg - get trial averages(s data, x smth, epoch, fs, 'x smth'); 

t epoch = ((-1):(1 / fs):(epoch - 1 - (1 / £s))); 

e cruda c (Mesa = al = (il / sei) )e (aa / ss) 2 (SD); 


%% plot graphs 


esp Pilote 


index - 
t index 
r index 
transp 


(0 


= 1)) 5 


CB eis o e 


:(K 

index i fs 
((K 

MOT 


subplot (611); 
pilote e S data. 


ylabel( 


set (gca,’ 
ylim([(min(s data.x) 


title(' 


State Estimation with 


ja e Da 
ey 


(you may need to press the Enter key again)'); 


o 


% reverse index 


ss, eolo y Glos / 255) O (204 / 265); 
epica salain TO e GITE 


[DE 
= nal) 


(max(s_data.x) + 0.1)]); 
Experimental Data’); 
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subplot (612); 


hold on; 

plot(find(n == 0) / fs, max(fp mode) * 1.3 x ones (length (find(n 
ONES MarserBacecolcon0 cc Maaicesi zoe 3) 

elor (ino (a Se 30) Y sey, max(fp mode) * 1.3 x ones (length (find (n 
-cedqDPEgscasMarkerRacecolor “gi “Markersize I 

plotteumdex m tpolmodere ))i; 

ylabel( i xr) 

xlim( [0 pui de MN ylim([0 (max(fp mode) * 1.5)1); grid; 


set(gca,'xticklabel', []); 


subplot(613); 


hold on; 
plot(tindex r rismtei a “cole”, 10 0.3 OT sre ws clt MET S 
plc (e moles, (rss o Siecle), “cales”, 10 0,0 01); 


ylabel (Me) e* {rm {ky} py) grid; 
xlim([0 t_index(end)]); 
set (gca, 'xticklabel', []); 


subplot (614); 


hold ou; 

[ole (e sales, 8 muc, 22", “ediles”, [0.5 (25 / 255) (66 / 255)1, 
"linewidth", 1.5937 

los ie imde 8 + Sres, “velez”, M O.5 (LYS / 28511); 


ylabel('(d) s_{k}’) 
xlim( [0 t_index(end) ||); grid; 
set (gca, 'xticklabel', []); 


subplot (615) ; 


bold om: 

plc (e moles, + Emiela, olo MEAE 

pillo (Geattavel (Ge ee 0) / ES), (min(x smth) = 0.5) « ones (length(find(I 
==) OND) ES Marikershace Color i uka mU Maa esq zo 3 

loe (vessel (ae ex 3D) Y ES, (min(x smth) - 0.5) « ones(length(find(I 
MAS SAN E Se Ra ceco lor mienim Markers ir eS 

ylabel (' x (k)'); ylim([(min(x smth) - 1) (max(x smth) + 1)1); 

xlim( [0 UR ae grid; label Time (Ss), 


subplob(6 loe 


hold on; 

plor le sooda, Ssa ena ll, T Lo) 9 

EUL Ee eysrexelev, exe eaa, Teka eve, $) Elo (elsa ee 93215 
'r', 'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 

pillo (e cuela, Elsa Ene, 2) “y basa, Eb 

Es (dle ceca, E Guaca, Elsa sas, 8) ndisse esa eyes, 99 
'm', 'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 

joule (e cueca, hsm Enel, eod on i ONSE mewsdth 


(t_ 
qos 
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eal! (le Guada, tess epeal, Teka eve, $) Eloisa el, 31 5 
lg, “BdgeCollor’, “none”, “EaceAlpha”., 10-2) 

xlim([t_epoch(1) t_epoch (end) ]); 

ylim([(min(min(skn_avg)) - 0.5) (max(max(skn avail + 0.5)]); 

grid; 


xlabel('Time (s)'); ylabel(’(f) z (k]'); 


subplot (6 Ep 2) y 


hold om; 

puke (E cuela, 31 Eme, 2), Yuet, abbsswabehrla , LoS) y 

calle Grecia, e _Cyorexela| be bw. 2) lolas enel, sy )il_, er 
'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 

pila (Ge cuela, oravit e rums “7 ilsliaveryaleheia’, LoS) 

cado (le Eyecia, Te epoch , be bw $) lola ENS Oo, Be) am 
'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 


plot (E cueca, 21 eme, 8), "xedexs". [0 0.6 0, Vibhaswbsitm",. 3L55) 5 


cad le Eyecia, Te (exexeXeled| y le ena, 8) Ellas ENS, silly "s 
'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 

xlim([t_epoch(1) t epoch(engd)]1) ; 

ylim([(min(min(x avg)) - 0.2) (max(max(x avg)) + 0.2)1); 

grid; 

xlabel('time (s)'); ylabel('(g) x (k]'); 


%% supplementary functions 


functdoHSvVE-Egetlposteriormede spreche 30, veil, 190, Jol, va, 
i, S, SO, El, wes) 


o 


M = 2007 % maximum iterations 


te = esos, i) 
i8 > Mara, s 
df = merosíil, My; 


abe (3D) es es joel) 
ie ab as Hyi = db) 
Ea) = ale (al) e se pree Co (uL se we s (e > 0 = m cs 
x pred) + sl x vr » (s - sO - sl « x pred) + 
Vr * VS x bl x (n- (1 / (1 + exp((-1) x» (bO + bl x 
tte: (05) IU B 


df(i) =1 + C « vr » vs x (bl ^ 2) « exp(b0 + b1 « it(i)) 
J| UG s exe(o@ = deu & de) ^ 2) 
) 


aie (sb s 3) e cse) = EU) / gh) 
if abs(it(i + 1) - it(i)) < 1e-14 
y e aie (al 45 il) ¢ 
return; 


end 
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end 
error ('Newton-Raphson failed to converge.'); 


end 
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function y = get maximum variance(z, r0, rl, W, x smth, K) 


(ro ^ 2) + (ri ^ 2) * sum(W) 
= 2 mom Eum = e seul O EAS MEA 
TO SU (as mE) ER; 


* 
* 


end 


function y - get linear parameters(x smth, W, z, K) 


zZz) +2 x 


y = [K sum(x smth); sum(x smth) sum (N) \ [sum(z); sum(z .x 


x_smth)]; 

end 

function y = get_trial_averages(s, x_smth, epoch, fs, opt 
y = zeros(9, epoch + fs); 


csm ep = zeros(length(s.csm), epoch «+ fs); 
Cspausmepe——zeros (lengths. Cspals)) epocha ts 
cSspenusmepE-EzenoslengbEbsscspanus)epoohw x fS» 


csm = s.csm; 
csp_us = s.csp_us; 
csp nus = s.csp_nus; 


si Slam opt lon “Es STI HD 


for j = 1:length (csm) 
(eim Cola), 2) = 2 sine ( stories (esit) = at) )8 
feu) um GS) ge es 3)$ 
end 


woe! Sp e ag Icxarsjelal (retsror ts) 
Cio WE CoG, 8) Sc omth TSTST EHH es pU) = 
stamcespsus() 9 x fs ED 
end 


Bora -asengihespanmus)) 
csp nus ep(j, :) = x smth((s.stim(csp nus(j)) 
s.stim(csp nus(j)) + 9 « £s - 1)); 
end 


elseif strcmp(option, 'skn') 


EOST RT STREET See 


ion) 


(s.stim ( 


ier) 8 (Es 
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end 
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cam Gola), 8) = Sox((Sosism(Csi(a)))) = is Eo stie Sm 
(339) +50: 8 = 10) 7 
end 


Eo Sp e tengais CE) 
Csi UES Sr, 3) = So le Stime CE = e eS. 
stamos prus (a) OE SO e 
end 


for j = 1:length(csp_nus) 


Go ms Soy, D = Soest sstumitespausi MEE SENS 
stim(csp nus(j)) + 9 * fs = 1)); 
end 
end 
wl; $) = meamleaa ta E), 
VE) mean (e spus 60) tative (034975 a hengt he spaus) = T 
std(csp us ep) / sqrt(length(csp us)); 
VIG mean (CSpeusmep) stan (OnO02 55 Hength(espmus)) T 


std(csp us ep) / sqrt(length(csp us)); 


y(4, :) = mean(csp nus ep); 

VITE EEmean(cspenusmep)NErStanvi09975pe'ensthicspenus) E 
* std(csp nus ep) / sgrt(length(csp nus)); 

WS, 3) 5 meses mis Ss) + ai (0.025, lencia (ces aus) = 1) 
* std(csp_ nus ep) / sgrt(length(csp nus)); 

wi, 8) = Wier (Cel ST eH 

ys, D = mean (comite) = t3mv(09975/ length (csm) E x std 
esm ep) / sqrt (length(csm)); 

WS), 8) Emeancesmeco)N + icahaie(@cO25, lerce ilee = al) <> siem 
csm ep) / sqrt (length (csm) ) ; 


10.4 State-space Model with One Binary, Two Continuous 


and a Spiking-Type Observation 


10.4.1 Simulated Data Example 


load('data one bin two cont one spk.mat'); 


delta - 0.005; 


$$ parameters 


5e5; $ maximum iterations 
= le-5; $ convergence criteria 


10.4 State-space Model with One Binary, Two Continuous and a Spiking-Type. . . 


bO = zeros(1, M); $ binary GLM model 

bl = zeros(1, M); 

rO = xerosii. M); % continuous model 

rl = zeros(1, M); 

Naz cr eene, MP % continuous model noise variance 
S0 = zeros(l, M); % continuous model 

sl = zeros(1, M); 

vs = zeros(1, M); % continuous model noise variance 
ve - zeros(1, M); $ process noise variance 

rho = zeros(1, M); % random walk fogetting factor 
alpha = zeros(1, M); $ input gain parameter 


K = length(n) ; 


X pred = zeros(1, K); 
v pred - zeros(1, K); 


Pe mere = Amos (Gl, dur 
MZ woche = eros EHE 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


p_updt = zeros(1, K); 


1 


Aeros FERK] 
Zeros I K) 
K 

K) 


= 
Il 


YE 


D 


CW = serosil, 
C= zeros 2, 


rpeaks = zeros(1, K x 50); 

rpeaks (round(rpeak_locs / delta)) = 1; 
rpeaks = reshape(rpeaks, [50, K])'; 
exception_counter = 0; 


$$ initial guesses 


base prob = sum(n) / length(n); 
b0(1) = log(base prob / (1 - base prob)); 


Sitios la 

r0(1) = 0.27154; 
rl(1) = 0.5057; 
Mae (Gab) = MOALE e 
s0(1) = -0.73899; 
s1(1) = 0.25324; 
WS) = 000302; 


ve(1) = 0.01883; 


155 
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rho) = 0.992143 
alpha(1) = 0.00818; 
theta - theta'; 

eta = -0 001; 


sg main function 
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for m= 1:M 
Por ke ek 
sag (e == 3) 
x pred(k) - x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
x pred(k) = rho(m) » x updt(k - 1) + alpha(m) * I(k); 
v pred(k) = (rho(m) ^ 2) * v updt(k - 1) + ve(m); 
end 
C(k) = v pred(k) / (vr(m) * vs(m) + v pred(k) * ((r1(m) a 
2) * vs(m) + (sl(m) ^ 2) « vr(m)); 
try $ numerical issues can occur due to the integrals 
[temp1, temp2] = get posterior mode(x pred(k), C(k), 
iB ae in) , seal tim), 1X9 (uU), do in), vic la), (ls), Gus) SO (n) 
s1(m), vs(m), 
rpeaks(k, :), ul(k, :), delta, w(k, :, :), theta 
', eta); 
x updt(k) = templ; 
p updt(k) = 1 / (1 + exp((-1) « (bO(m) + b1(m) x 
x updt(k)))); 
xz verte (is) e 3b / (Gb 7 y cata) = (Eloy ^ 2) / vel 
m)) + ((sl(m) ^ 2) / vs(m)) + (bi(m) ^ 2) « p updt(k) * (1 - 
p updt(k)) - temp2); 
catch 
x updt(k) - x pred(k); 
v updt(k) - v pred(k); 
exception counter = exception counter + 1; 
end 
if (mod(k, 100) == 0) 
EpaumttE eda o 
end 
qt modice 2500) == 0) 
arcas (7 Nast? )) m 
end 
end 
x smth(K) - x updt(K); 
v smth(K) - v updt(K); 
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W(K) = v smth(K) + (x smth(K) ^ 2); 
Me (emel = 3D) = mao) «s y Most Ge (sae) = 35) m v pred(2:end 
DE. 
for ko UK 13-10) 21 
x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - x pred(k + 
IDDE 
v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 
v pred(k + 1)); 
CW(k) = A(k) + v smth(k + 1) + x smth(k) + x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
end 
if (m « M) 
Ra-Egetmiumearnapanrametessiouemth m WEISE 
S - get linear parameters(x smth, W, s, K); 
OQ) = sumisa! e 35) (at (2ecine)) xx se eias (eroi = 35) 7) 7 
(I(2:end) * x smth(1:(end - 1))') (I * I')] Y [sum(CW 


); (I(2:end) « x smth(2:end)')]; 


( 


* 


b0(m + 1) = log(base prob / (1 - base prob)); 
biim 1) = 1; 
seen d» 3D) = (Ni. i); 
sie (ONE, 3). (9) $ in case this happens (generally 
only needed with experimental data) 
alpha(m + 1) = alpha(m); 
else 
alpha(m + 1) = Q(2, 1); 
end 
ve(m + 1) = (sum(W(2:end)) + (rho(m + 1) ^ 2) * sum(W(1: 
end - 1))) - 2 x rho(m + 1) * sum(CW) - 
2 * alpha(m + 1) + (I(2:end) + x smth(2:end)') + 2 
alpha(m + 1) « rho(m + 1) « (I(2:end) « x_smth(1: (end - 1))') 
+ 
(alpha(m + 1) ^ 2) « (I * I')) K 
Om > 3D) = Ra, Y, 
ri(m + 1) = R(2, 1); 
SO (im c 3) e Sui 1), 
Slim 1) 80] 13 
vr(m + 1) = get maximum variance(r, r0(m + 1), rl(m + 1), 


W, x smth, K); 


vs(m + 1) = get maximum variance(s, s0(m + 1), sl(m + 1), 


W, x smth, K); 
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mean dev = mean(abs([bO(m + 1) b1(m + 1) rO(m + 1) Cl (m 
+ 1) ve(m + 1) vr(m + 1) rho(m + 1) alpha(m + 1) sO(m+ 1) s1 
(m + 1) vs(m + 1)] - 
[bO (m) b1(m) rO(m) r1(m) ve(m) vr(m) rho(m) alpha (m) 
s0(m) s1(m) vs(m)])); 


if mean dev < tol 
fprintf(’\n\nConverged at m = %d\n\n’, m); 
break; 
else 
fprintt (i) cse Echo) cx Solas c bT ENNO e E TZ 
Vnde ss defun = e lef\ninso es lStNusq = 4. Seas = 3.18 
fNnNnve = %.18£\nrho = $.18f£\nalpha = %.18£\n\ndev = $.T18f£NnN 
m 
mod, bon ck D. bLu + EL. rom x 32), El 4 Li, we 
(m+ 1), sO(m + 1), slim + 1), ve(m + 1), vem + 1), rho(m + 
1), alpha(m + 1), mean_dev) ; 


se joel = ewe 19) 7 
voce >= magos (a, 19 7 


oe iherche = moron MEKE 
y Moe = mera OPEKE 


sie mca acu) c meros, mK = 3D)R $ x smth(1) 
needed for next iteration 
y amda = zeros 10) 7 


D wycie = meros (al, 19) 7 


A = zeros(1, K); 
W = zeros(1, K); 
Ig 

K); 


D 


CW = zeros(l, 
Cos zerosgsit. 


end 


%% calculate confidence limits 


p smth = 1 ./ (1 + exp((-1) + (bO(m) + b1(m) * x smth))); 
r smth = r0(m) + rl(m) * x smth; 
s smth = s0(m) + s1(m) + x smth; 


lambda - zeros(K, 50); 
mean rr - zeros(K, 50); 


ope ab ex LRK 
iue 3 ex abs 


wl = [squeeze(w(i, j, :))’ [eta x_smth(i)]]; 
if (f(theta', ul(i, j), w1) » 1e-18) 
lambda(i, j) - fetch lambda(theta', ul(i, j), w1); 


end 
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nea soli, 3) = muleta”, a), 
end 


lambda start index = find(reshape (rpeaks', 1, numel (rpeaks)), 1); 
lambda = reshape(lambda’, 1, numel(lambda)); 


IL = get log I TRT G T speaks, ul, Celte, y) theta y +5 Emen 
g v smth), 

11 final - sum(nansum(11)); 

$$ plot graphs 

figure; 

mean rr - reshape(mean rr', 1, numel(mean rr)); 

neve, car (posee LOSE) y 


rr times = rpeak locs(2:end); 


index = (0:(K - 1)); 


fs_ hyp = 4; 
t_index = index / fs_hyp; 
rendez L = al) g((e3D) 80) 2 reverse lacteos 


transp = 0.35 


subplot (611); 

hold on; 

olle (e acle, T m emic 
loe (fe moles, D Gwen, 7227) p 


( 
( 
polset Carasa == 0) = 30) Y fs hyp, 1.2 * max(p) * ones (length (find 
(NAS ksi V MarkernaceCcolor n ki, Markersizeu A) 
plot((find(n == 1) - 1) / £s hyp, 1.2 * max(p) * ones(length(find 
(n == 1))), ‘gs’, 'MarkerFaceColor', 'g', 'MarkerSize', 4); 
AE AO DA 
title('State Estimation with Simulated Data'); 


subplot (612) ; 

hold on; 

poulkoye (ie aliayolope, ae, End 
pilori inder we mw Cp 
ylabel('(b) r (k)'); 


subplot (613); 

hold ou; 

¡loe (e acts, (my 71973) Erd; 
olore taces; SES mt 
yabe cS ME T 


subplot(614); 


hold on; 

E (te sales, se ae), ard; 

pilotes, ee Emela, 26" e 

pila Gieh == y = 3) / fs hyp, (-8) * ones (length(find(I == 0) 


)), ‘ks’, 'MarkerFaceColor', 'k', 'MarkerSize', 4); 
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plot((find(I == 1) - 1) / £s hyp, (-8) * ones(length(find(I == 1) 
)), ’es’, 'MarkerFaceColor', 'c', 'MarkerSize', 4); 
ylabel Cd) x (ky; 


subplot (615); 

hold on; 

plot (rrieimes eri “ey, color lil, 05, 02, 7 
MavkerbaceColon 1017207550925] MarkersSize! 7 72) 72 grid: 

mu start index = round(rpeak locs(2) / delta); 


plot(((0:(length(mean rr(mu start index:end)) - 1)) * delta), 
mean rr(mu start index:end), 'b'); 
ylabel('(e) rr (i]'); xlim([O t index(end)]); xlabel('time (s)’); 


subplot (616); 

qqplot(x smth - x); grid; 

title(’QQ Plot - State Estimate’, 'FontWeight', 'normal'); 
ylabel('(f) input quantiles’); 

xlabel('standard normal quantiles'); 


figure; 
get ks plot (rpeak_locs, lambda(lambda start index:end), delta, 1) 


ylabel({’Theoretical’, 'Quantiles']); xlabel('Empirical Quantiles 
De 
title(^KS Plot): 


$$ supplementary functions 


irravenesLore (by BA = ger poterlo nocle (bs jewel (E, 325 320, wil, 190, 
Isl, Wit, my E, SO, (ub, YE, sires, ul, celta, vy ell, iae 
eta) 

M = 200; % maximum iterations 


IZ SOS CEN g 
func = zeros(1, M); 
diriS- ao) ME. 


abi (UL) 


x pred; 
moja ab = ae Qui = dL) 


isl, m» Paros (il, SOs 
H2 = zeros(1, 50); 


worm y = dis = Sms) =>) 0.25) si (4) hz, for skin 
conductance) 
Wo eeuu. cul (al, 3; 8) )% liie abs (st) 11) 9 


if (f(theta, ul(j), w) > 1e-18) $ 
lambda - fetch lambda(theta, ul(j), w); 
dl dx - dlambda dx(theta, ul(j), w); 
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H1(j) = dl dx x (rpeaks(j) - lambda + delta) / 
lambda; 
H2(j) = d21ambda dx2(theta, ul(j), w) * (rpeaks(j 
) - lambda + delta) / lambda - rpeaks(j) + (dl dx EET 
lambda ^ 2); 
end 
end 
Hl = sum(iHi); 
H2 = sumi(H2); 
cum (a) e» de (ai) = xe jas) = ES (seal ve: (G9 = 3289 = ial oc 
es gosse) + fal «s Was x (sa = m = El c se joel) + 
vr » VS x bl x (n- (1 / (1 + exp((-1) * (bO + b1 x 
sb GOOO d NAS o Aides > Ob) S 


df(i) = 1+ C « vr » vs x ((b1 ^ 2) « exp(b0 + bl « it(i) 


) Y (GE ss sl ss toil = het). ~ 2) e 22); 
aie (al s 3) e. chen) = muela) / que (a) e 
if abs(it(i + 1) - it(i)) < 1e-14 
EE (ak dp Gl) y 
return; 
end 
end 


error('Newton-Raphson failed to converge.'); 
end 


function y = get maximum variance(z, r0, rl, W, x smth, K) 


y= (z « z’ + K « (r0 ^ 2) + (x1 * 2) + sum(W) 
= 2 c pO i sun) = 22 wil Clot (fs emen, A) + Bs 
Oy cl su(s men) KR; 


end 
function y = get linear parameters(x smth, W, z, K) 


y - [K sum(x smth); sum(x smth) sum(W)] \ [sum(z); sum(z .« 
x smth) 1 : 


end 
function [y] = f(theta, t, w) 
y = peace (thetalsnd) 7 (2 2 iol xs Ce 27 $90) 59 0. 
exp((theta(end) « ((t - mu(theta, w)) .^ 2)) ./ 
((-2) + (mu(theta, w) ^ 2) * t)); 


end 


function [y] = intf(theta, t, w) 
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y = integral (0 (t)f (theta, t, w), 0, t); 
end 
function [y] = mu(theta, w) 

eta = w(end - 1); 


x = wend) ; 
p = length(theta) - 2; 


Il 


y 
end 
function [y] = fetch lambda(theta, t, w) 


cdt > alista Cheta y Ww) B 
y e Eldar, E, YD o/ (L e ee) 


aE lede > 1) % numerical issue 


end 


function [y] = df dmu(theta, t, w) 


MATLAB Code Examples 


theta(1) + theta(2:(2 + p - 1)) x w(1:p)' + eta x* x; 


y = (theta (end) (miele, w) * 999) = (Ge(idiaciec, E, v9) ac (e 


- mu(theta, w))); 
end 
punet one yl = Che dehet ic, vi) 


eta = w(end - 1); 
y e Che chimie ley V) ot Stay 


end 
sueo NA] = mech Chilenska; Ey w) 
y > necesa (oli) cha cara, Es Wy 0, Elz 
end 
function [y] = dlambda_dx(theta, t, w) 
cdf = inttítheta, t, w); 


ia Keske z il) % numerical issue 


ye (Cl = ek) a ele es aasica, E, Wy) + 
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end 


function 


x (elveel, Er W) ses iaeei Choltice, te, W) o// (UL = glz 


DA = cer cava (acera, Ey SW) 


y = theta(end) « (df dmu(theta, t, w) .* ((t - mu(theta, w)) 
/ 


(mutet mw) mS) NE 
f (theta, t, w .* ((2 » mu(theta, w - 3 x t) / (mu(theta 
o DANNA 
end 
function ly] P=tazEtd=2 (theta Cw) 
Gra = weno = 1), 


y = d2f dmu2(theta, t, w) .* (eta * 2); 


end 


function 


[y] = intd2f dx2(theta, t, w) 


y = integral(G(t)d2f dx2(theta, t, w), 0, t); 


end 


function [y] - d21ambda dx2(theta, t, w) 


ye (4 s clane cheisacice, Ey v) x» (al = dimer (ara, Uy W) E 
intdf dx(theta, t, w) + 

(lee Cha lacra) i, Wu c (ab c» sbaweaslürlwciecu, Ey WI) cR 56 

f (theta, t, w) « intd2f dx2(theta, t, w)) / ((1 - intf( 


theta, 


end 


En DI De 


function [y] = get_log likelihood(eta, rpeaks, ul, delta, w_all, 


theta, x, v) 
K = length(x); 
y = zeros(K, 50); 
Por ee 
were gp e 350 
We [Hexe ula (e, 35 20) erca ss 115 


if (f(theta, ul(k, j), w) » 1e-18) 


lambda - fetch lambda(theta, ul(k, j), w); 
dl dx - dlambda dx(theta, ul(k, j), w); 

d21 dx2 - d21ambda dx2(theta, ul(k, j), w); 
nkj = rpeaks(k, j); 
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y(k, j) = nkj * log(delta « lambda) 
lambda + 


- delta x 


(d21 dx2 » (nkj - lambda * delta) / lambda - 


ale) c («el ks ^ 2) / (ülemdselt ^ 2) e ws) > 0.57 
end 


end 
end 


end 


10.4.2 Experimental Data Example 


load('expm data one bin two cont one spk.mat'); 


delta - 0.005; 

min scr thresh - 0.015; 
min scr prom - min scr thresh; 
fs = 47 

Guaca = 1107 


stim = s data.aug stim; 
ph = s data.ph; 
tn = s data.tn; 
rpeaks = s data.rpeaks; 
ul = s data.ul; 


[pks, locs] = findpeaks (ph, 'MinPeakHeight', min scr thresh, ' 


MinPeakProminence', min scr prom); 


r = interpl([1 locs length(ph)], log([ph(find(ph > 0, 1)) pks ph( 


end tengetn((sidata pa) ense 


s e ages 
zeros(1, length(r)); 
I = zeros(1, length(r)); 


sy 
I 


n(locs) 
I (stim) 


1; 
1; 


Il 


SES m e secs) $ 
peel se es sucia) p 


Sse / eus 
me ae f guo ip 


$$ parameters 


M = 5e5; $ maximum iterations 

tol = 1e-6; % convergence criteria 

fo) =S edos (PEME $ binary GLM model 
bl = zeros(1, M); 
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rO = xeros(i, M); % continuous GLM model 

vel, c» vehe dis Ag 

Vie = zeros (i), M) % continuous GLM model noise variance (1) 
S0 = zeros(1, M); % continuous GLM 

Sil S euer 10) fp 

vs = zeros(1, M); % continuous GLM model noise variance (2) 
ve - zeros(1, M); $ process noise variance 

rho = zeros(1, M); % random walk correlation 

alpha = zeros(1, M); $ input gain parameter 


K - length(n); 


be joel = memos (il, gs 
vepred- emos (il, 19) y 


x_updt = zeros(1, K); 
W owale = zeroki, I; 


x_smth = zeros(1, K); 
Wo Emela = zeros al, 1) y 


jo) use. = ams ll, 19) y 


D 
[i 


zeros(1, K); 
Ex erosi MERKE 
CWE- Peñon, 19 
E cx mesos, 197 


1 


= 
I 


i 
exception_counter = 0; 
$$ initial guesses 


is prob - sum(n) / length(n); 
0(1) = log(base prob / (1 - base prob)); 


) 

ae (ab) dl 

0(1) = r(1); % guess it's the first value of r 

a) m abe 

0(1) = s(1); 

a) ex de 

Clb = 006p 

SHH = 00057 

ell) = 0.05; 
Eon = ily 
alpha(1) = 0.5; 


theta = s data.theta; 
etam = SO Oil; 


s% main function 
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if (k == 1) 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
x pred(k) = rho(m) « x updt(k - 1) + alpha(m) x» I(k); 
v pred(k) = (rho(m) ^ 2) * v updt(k - 1) + ve(m); 
end 
C(k) = v pred(k) / (vr(m) * vs(m) + v pred(k) * ((r1(m) ^s 
IS E Sela m) AO 
Ey $ numerical issues can occur due to the integrals 
[temp1, temp2] = get posterior mode(x pred(k), C(k), 
ii) O slid), 0 a), lel (n9) p va), ms), S) n BO (0) , 
s1(m), vs(m), 


moss, eB), Cul, 8), Celte, e cearca we By 8), 
theta', eta); 
x updt(k) = templ; 


p updt(k) = 1 / (1 + exp((-1) « (bO(m) + b1(m) x 
x updt(k)))); 
W_Wische (ie) e x / (1 / y predio) e (Ge) ^2) / ws 
m) EIS ^ 2) / seu) = Cal) 2) « p updt(k) « (1 = 
p updt(k)) - temp2); 
catch 


x updt(k) - x pred( 

v updt(k) - v pred( 

exception counter - 
end 


ie Mocis, 100) == 0 
a (Il PEE 


end 
if (mod(k, 2500) == 0) 
TGT c (0 iato )) 9 
end 
end 
x smth(K) - x updt(K); 
xc smc (Ed) = we gel (1) $ 
W(K) = v smth(K) + (x smth(K) ^ 2); 
Me (emel = 3D) cx sema) x we Heal EL R (us = 35) n v pred(2:end 
DE, 
for ka ke = dd 


x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - x pred(k + 
1)); 
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v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) 
v pred(k « 1)); 
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CW(k) = A(k) + v smth(k + 1) + x smth(k) +» x smth(k + 1); 
(k) 


W(k) = v_smth(k) + (x_smth(k) ^ 2); 


sie (il e MI) 


R = get_linear_parameters(x_smth, W, r, K); 


S = get linear parameters(x smth, W, s, K); 
(9) c sumisa = 35)) (asen) ey se micas (eio! => 359) ) 5 
(I(2:end) * x smth(1:(end - 1))') (I * I')] Y [sum(CW 
)g (Bec a H 0 (2 sc) "J| 5 
bb = fsolve(@(b) binary parameter derivatives(b, n, 
es [tay we Erie), SS alll, opti meet (isla) V exea JR 
b0(m + 1) = bb(1); 
bi (m + 1) = bb(2); 
imei ss 3.) = Ou. 1), 
JE Ora c Ey a o $ check in case this happens 
alpha(m + 1) = alpha(m); 
else 
alpha(m + 1) = Q(2, 1); 
end 
ve(m + 1) = (sum(W(2:end)) + (rho(m + 1) ^ 2) + sum(W(1:( 
end - 1))) - 2 x rho(m + 1) + sum(CW) - 
2%) alpha(m + E (T2: end) x x smth(2:e0d) E20. 


colla (im) x 3D) <3 cao se 30) x en y ss Ema (le (wel = 3D) 7) 


+ 
(etanata = 3) ~ 99 2 (uo x73) / xg 
ie absgetumazmumsvardiance(r RTT RN 
xNsmteh MK) get maximům variance (Sh SCOM PS, 
ss amela K) > Ocal) c Etenmimadtesoncesovertittumdgeissdececteci 
break; 
else 
r0(m + 1) = R(1, 1); 
r1(m + 1) = R(2, 1); 
SO Gm + 1) = 86, E); 
cla e 3D) = EM DD), 
vr(m + 1) = get maximum variance(r, rO(m + 1), r1(m + 
3D) 5 Wy sx mw, 10) y 
vs(m + 1) = get maximum variance(s, sO(m + 1), s1(m + 
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mean dev = mean(abs([bO(m + 1) b1(m + 1) rO(m + 1) Cl (m 
+ 1) ve(m + 1) vr(m + 1) rho(m + 1) alpha(m + 1) sO(m+ 1) s1 
(m + 1) vs(m + 1)] - 
[bO (m) b1(m) rO(m) r1(m) ve(m) vr(m) rho(m) alpha (m) 
s0(m) s1(m) vs(m)])); 


if mean dev « tol 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
aora m cse Echo) = SedlbENslenh cs Ele ENNO Cc 2 
Mns defunt = e d8tNnuus0 es lStNusq = %7. der\nve = 3.18 
fNnNnve = %.18£\nrho = $.18f£\nalpha = %.18£\n\ndev = $.T18f£NnN 
m 
mod, bon ck D. bLu + EL. rom x 32), El 4 Li, we 
(m+ 1), sO(m + 1), slim + 1), ve(m + 1), vem + 1), rho(m + 
1), alpha(m + 1), mean_dev) ; 


se joel = ewe 19) 7 
voce >= magos (a, 19 7 


oe iherche = moron MEKE 
y Moe = mera OPEKE 


sie mca acu) c meros, mK = 3D)R $ x smth(1) 
needed for next iteration 
y amda = zeros 10) 7 


D wycie = meros (al, 19) 7 


A = zeros(1, K); 
W = zeros(1, K); 
Ig 

K); 


D 


CW = zeros(l, 
Cos zerosgsit. 


end 


calculate confidence limits 


p smth = 1 ./ (1 + exp((-1) * (b0(m) + bl(m) » x smth))); 
dkeuL se) = aos (al, dE 
ell depo c er (El, 1) y 


r smth = exp((r0(m) + x1(m) * x smth) * std r); 
s smth = (s0(m) + s1(m) * x smth) * std s; 


Skn avg = get trial averages(s data, x smth, epoch, fs, 'skn'); 
X avg - get trial averages(s data, x smth, epoch, fs, 'x smth'); 


t epoch = ((-1):(1 / fs)s(epoch - Y - (1 / fs))); 


EER 


cunda = (evada = t= (GL / Es))s (sl / zels (DD) y 
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RSE Da S (Ba Nut? Jg 


lambda - zeros(K, 50); 
necia sa > Eos, 510) y 


ojal = abe ne 
zer gJ = 250 


vw = lEquesrale cara, aj, 8)) [Ga suela (Gt) 1) I 7 
e aces, Vat, 1). 1) => lesils) 
lambda(i, j) = fetch lambda(theta', ul(i, j), w); 
end 
mean_rr(i, j) = mu(theta’, w); 


end 
end 


lambda start index = find(reshape(s data.rpeaks’, 1, numel(s data 
.rpeaks)), 1); 
lambda - reshape(lambda', 1, numel(lambda)); 


11 = get log likelihood(eta, rpeaks, ul, delta, s data.w, theta’, 
x smth, v smth); 
11 final = sum(nansum(11)); 


%% plot graphs 
mean rr - reshape(mean rr', 1, numel(mean rr)); 


Dru Edo EEsmdataenpeakmlecsy) 
rr times - s data.rpeak locs(2:end); 


index = (0:(K - 1)); 
t index - index / fs; 
je alas c» (HS = al) (SL) $0) 7 


transp - 0.3; 


subplot (711); 


jolie (ie neces, e caras: “Soler”, drag / 255) O (202 / BES) I) E 
ylabel('(a) z (k)'); grid; xlim([O t_index(end)]); 
set(gca,'xticklabel', []); 

ylim([(min(s data.x) - 0.1) (max(s data.x) + 0.1)1); 


title('State Estimation with Experimental Data'); 


subplot (712); 


hold on; 

nilo (Gracia == 0) / ds, max(p smth) * 1.3 * ones (length (find (n == 
0 test MarkerkaceCobor "k^" MarkerSaize 3); 

[obse (iaa exc 3D) Y ES, max(p smth) * 1.3 « ones(length(find(n == 
1)));, gs, "MarkerFaceCoLlor'^, "q^ /MarkerSaze* | 3); 


piottaimcdexoemti MEN 

ylabel('(b) p (k)'); 

xlim( 10) t_index(end)]); ylam( [0 (max(p smth) * 1.5)1); grid; 
set (gca,’xticklabel’, []); 


subplots 3) 
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hold on; 
plot (tiindex ri ismeh Moe 10 0.3 01, “insista, das), 
pollo (E amos, epale Siecle), “cole”, 10 0,8 Ol» 


ylabel(“(e) eek) grid; 
xlim([0 t_index(end)]); 
set (gca, 'xticklabel', []); 


subplot (714); 


hold on; 

jolie (e dades, 8 mdr, "2^, "eoe", [0.5 (25 / 255) (66 / 255) 1, 
‘linewidth’, 1.5); 

loe dades, 8 v» seee, “soler”, [fub 0.5 (uwe / 285811) 7 


ylabel('(d) s [k)'); 
xlim([O t_index(end)]); grid; 
set(gca,'xticklabel', []); 


subplot (715); 


hold on; 

pole (de sele, +2 Emela, “alar”, 19%) 7 

¡lo (ala == 0) / Sy (min(x smth) - 0.5) * ones (length (find (1 
== 0))) ES) n MarkerkaceCotor kk? “MarkerSizes 3) 

plot (find(I == 1) / fs, (min(x smth) - 0.5) * ones (length(find(I 
== 111), “es”, 'MarkerFaceColor', "e", 'MarkerSize', 3); 


ylabel (' (e) x (k)'); ylim([(min(x_smth) - 1) (max(x smth) + 1)1); 
ma (0) ie _aliatelepe(Ciarel)) I T cts clellatoed STT 


subplot (716) ; 


hold on; 

[ole (ie Times / GO, wel, “9, "Gees", IL, 0.5, 0.25], " 
Markersacecolonc IM, 0-5), 0.251) Markensize' 3); grid; 

mu Start index = round(s data.rpeak locs(2) / delta); 

plot(((0:(length(mean rr(mu start index:end)) - 1)) « delta) / 
60, mean rr(mu start index:end), 'b'); 

ylabel('(f) rr (i)'); xlim([O t index(end)] / 60); xlabel('time ( 
min), 


ano e 2, E) 


hold on; 

plc (E cueca, hsm Eng, e Eod ox SONO SOIN mewsdth 
1.533 

Es (le Grecia, Exe Guaca, esa ae, $) Else (reuse enel, 9315 
^q', "EdgeColor', ‘none’, 'FaceAlpha', 0.2); 

plc (e cueca, hsm ene, 8), Val’, Clasica, 5E 

ce Grecia, e Guada, lesa eNe, $) migo (Elsa ENS, y 
cm, “Edgelolor”, “none”, Facealpha!, 0.2) 

pila (E _Cysrexelal,, Elsa Enel, 3) 5, 4 eystlt e LaS) 

cada le aeda, te ceda, Esa eva As 2) logos (elsa I a) Ip 
x EdgeColor', ‘none’, “Facealpha”, 0.2) 
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xlim([t epoch(1) t epoch(end)]) ; 


ed 
xlabel('time (s)'); ylabel('(g) z_{k}’); 


Euler, 2 Ma) 


hold on; 

loe Wu Socia, Fx ewe, E), “color”, 10 0.8 01, ean EET 

cada le Geda, tesa Gaal, ls ewe $) eloa ewe e "sp 
'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 

plore (Ve Guada, + melts 8) Vi, inmensa, al,5))9 

cade Grecia, e cual y le chus o 2) mulas enel, si) il, "mm; 
'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 

plot (fe cuerda, Fx enel, 2) "E “ llaliaealcleia’ , .S)7 

cada (e eyecia, te ceda, ls chus 8) elioas Ex SS 8) )Il_p "387. 
'EdgeColor', ‘none’, 'FaceAlpha', 0.2); 


xlim([t_epoch(1) t_epoch(end)]); 


gerdi 

xlabel('time (s)'); ylabel('(h) x_{k}’); 

figure; 

get ks plot(s data.rpeak locs, lambda(lambda start index:end), 
delta, 1); 


$$ supplementary functions 


Eunccdon MH Eget. postierionamedeoopmsedo e E, 15, 30, sl, 190, 
leui, wat, i, 3, EW, El, YE, osea, Wil, celta, w T iate. 
eta) 

M = 200; % maximum iterations 


alie, e» EROS, IM) 
func - zeros(1, M); 
df - zeros(1, M); 


abe (b) 


x pred; 
moje ab Es ale (Qui = di) 


Hl = zeros (tl, 50) 
H2 = zeros (1, 50); 


wore gp e 1350 $ 5 ms => 0.25 8 (4 Hz for skin 
conductance) 
wa-esqueezeiwaas A Sta itii 


if (f(theta, ul(j), w) > le-18) $ 
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lambda = fetch lambda(theta, ul(j), w); 


dl dx = dlambda_dx(theta, ul(j), w); 

H1(j) = dl dx « (rpeaks(j) - lambda * delta) / 
lambda; 

H2(j) = d21ambda dx2(theta, ul(j), w) * (rpeaks(j 


) - lambda + delta) / lambda - rpeaks(j) + (dl dx ^ 2» 7 f 
lambda ^ 2); 
end 
end 


Il 


H1 sum (H1); 
H2 = sumiH2); 


cmo (a) = dela = ja. G c (Geil s we (ue = 20 = El e 
e jones) + fal c Was xs qb = 80 = El cx Ee jowecyel)) x 
vr + VS x bl x (n- (1 / (1 + exp((-1) * (bO + bl x 
it(i)))))) + vr * vs * H1); 


df(i) 2 1 € € « vr L ^ 2) * exp (b0 A A E) 


) 7 UG cl = toil EL TTT ^ 2) e sb) 
aie (ak se 3D) e mel) = science (Sb) / che (sb) y 
if abs(it(i + 1) - it(i)) < 1e-14 
y m aie (aL ab 3) 5 
return; 
end 


end 


error ('Newton-Raphson failed to converge.’) ; 


function y = binary parameter derivatives(b, n, x smth, v smth) 


end 


Y e vase 8) 
K = length(n); 


o 
o 
ll 


DIDE 


for ie = Iais 

p(k) 1 / (1 exp((-1) * (bO + bl + x_smth(k)))); 

y(1) = y(1) + n(k) - p(k) - 0.5 « v smth(k) « (b1 ^ 2) « 
pos c (Gl = jos) o (GU 2 & joe) )s 

AZ) EA a) Es Mee) SES ME c) xp) 
v smth(k) + bi * p(k) + (1 - p(k)) * (2 + x smth(k) + bl + 
(1 - 2 p(k)); 
end 


function y = get maximum variance(z, r0, r1, W, x smth, K) 
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^ 


y= [(z* 2" + K (ro 2») s (EL ^" mp e pita) 
= 2 x c Sun) = 2 x» wil & dor (e Emula, d) 2 Ae 
r0 * rl * sum(x smth)) / K; 


* 
* 


end 
function y - get linear parameters(x smth, W, z, K) 


y = [K sum(x smth); sum(x smth) sum(W)] \ [sum(z); sum(z .x 
x smth)]; 


end 
function y - get trial averages(s, x smth, epoch, fs, option) 
y = zeros(9, epoch x fs); 


csmaepesezerosilengshsscsmEepcche sets) 
Cspausmepe= zeros (length (Ss Cspmus)) i epoch + ts) 
csp nus ep = zeros(length(s.csp nus), epoch x» fs); 


csm = s.csm; 
csp_us = s.csp_us; 
csp nus = s.csp_nus; 


WE Strempl option MN St hd 


for j = 1:length(csm) 
csm ep(j, :) = x smth((s.stim(csm(j)) - fs ):(s.stim( 
eun) 42 G sS 39)5g 
end 


oe Sp ena (sp aus) 
Gao e an, 8) = Ema (e cla lea 10609)))) = ES) 8 (5. 
Srta S s S) oe = 3D) B 
end 


for j = 1:length(csp_nus) 
csp nus ep(j, :) = x smth((s.stim(csp nus(j)) - fs): ( 
Sstedimi(espmusi aR ST ti MEE 
end 


elseif strcmp (option, 'skn') 


for j = 1:length(csm) 
coma. 8) = mosso (3))) = ES 3 (Sosa (sim 
GDI +39: de) = dd), 
end 


"one eat sheng (aao wE) 
Gag WES GG, 2) = EMS sitesi (Sey CE = scl) (si 
StamicspmusiG) gee cS MEME 
end 
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for j = 1:length(csp_nus) 


Cao mws Coty, D = Sar (Sosa (Sejo aii DENEN 
stim(csp nus(j)) + 9 «+ fs - 1)); 
end 
end 
VTA -EmeancgpSu sp 
VI TEEDE-Emecan(cespsusmep)NE Sta mvlt0moy5n'engt h(cspsus) = aly) + 
std(csp us ep) / sqrt(length(csp us)); 
Wa, E) = meme) e Es + tumviq025peltengt i ospsis Ne E 
std(csp us ep) / sqrt(length(csp us)); 
y(4; - mean(csp nus ep); 
WS, 8) = meses) mue Gs + el (0.075, Jena (es ue) = al) 
* std(csp nus ep) / sqrt(length(csp nus)); 
WS, 3) = memes mus Ss) < iv (0.025, lenene aus) => db) 
* std(csp nus ep) / sqrt(length(csp nus)); 
WT, 8) x meanlcan_ (i9) 5 
SA, $) = mesi E) <P br (0,05, lance) = al) > Elec 
csm ep) / sqrt(length(csm)); 
AO, 8) = peenem ao) + ieahmie(@ 025, enee aee = a) siem 
Gam epl / sqrt (length(csm)); 


end 
function [y] = f(theta, t, w) 
X e ct (Basta (Gmc) .f (B sgh e (E. 2))) 28 00 
exp ( (theta (end) « ((t - mu(theta, w)) .^ 2)) ./ 
((29) s (u(tasea, uw) * 2) ou) 
end 
function [y] = intf (theta, t, w) 
y = integral (0(t)f (theta, t, w), 0, t); 
end 
function [y] = mu(theta, w) 
eta = w(end - 1); 


x = w(end); 
p = length(theta) - 2; 


y theta(1) + theta(2:(2 + p - 1)) x w(1:p)' + eta x* x; 
end 


function [y] - fetch lambda(theta, t, w) 
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cdti- mer (rasca, E) We) 
x? e ase, Es YY) ol (il = ex) 


ab (ehe > i) $ numerical issue 


end 
function [y] - df dmu(theta, t, w) 


y = (theta (end) (mu(theta, w) ^ 3)) * (f(theta, t, w) .* (t 
- mu(theta, w))); 


end 
function [y] - df dx(theta, t, w) 


eta - w(end - 1); 
y => Che Clinbi (icles, Ey Wh) ot Sta, 


end 
function [y] = intdf dx(theta, t, w) 
y > alinecepeeul (O (ie) che co lidiGca, E, WW), Op El 
end 
function [y] = dlambda_dx(theta, t, w) 


cat e inttítheta, t, w); 


AP edt: => ad) % numerical issue 
y = 0; 
else 
ye (al e esi) om ChE eb E, Ww) + 
(ee, T a imeeli Gba) ley W) of (b = cele 
) .* 2); 
end 


end 
function [y] - d2f dmu2(theta, t, w) 


y = theta(end) « (df dmu(theta, t, w) .« ((t = mu(theta, w)) 
/ 


(mu(theta, w) ^ 3)) « 
f(theta, t, w .* ((2 * mu(theta, w - 3 x t) / (mu(theta 
A 


end 
function [y] = d2f dx2(theta, t, w) 


Gea = Wien > 1) 
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yf = CH Chie meca, Es WW) so (Gea > 2); 
end 
cunc lony E > ZRH E (eba2 (clic, NEW) 
y = integral (@(t)d2f dx2(theta, t, w), 0, t); 
end 
function [y] = d2lambda_dx2(theta, t, w) 
ws (a & GClileinigck eb aria, Ej vy) vs (al = aer (cle, Ey W) 5 
intdf dx(theta, t, w) + 
d? tede? beta p W a G etf (theta, de) w) 
f (theta, t, w) x intd2f dx2(theta, t, w)) / (( 
theta, t, w)) ^ 2); 


end 


function [y] - get log likelihood(eta, rpeaks, ul, delta, w all, 
theta, x, v) 


A 
[ 


= length (x); 
Y = Zeros (KE, 510); 


ichs he = iais 
inoue 9) ex 1950 
w= leurs ela (ic, aj, 20) erca stiis 


if (f(theta, ul(k, j), w) > le-18) 


lambda = fetch lambda (theta, ul(k, j), w); 
dl dx = dlambda_dx(theta, ul(k, 3), w); 

d21 dx2 = d2lambda_dx2(theta, ul(k, j), w); 
nkj = rpeaks(k, j); 


y(k, j) = nkj * log(delta x lambda) - delta + 


lambda + 
(d21 dx2 +» (nkj - lambda + delta) / lambda - 


nkj « (dl dx ^ 2 / Gama ^ 2) e ws) & 0.57 
end 


end 
end 


end 
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10.5 State-space Model with One MPP Observation 

10.5.1 Simulated Data Example 

load('data one mpp.mat'); 


K = length(n) ; 


joe = Einen zx (9) 


M = Bed: 

ve = zeros(1, M); % process noise variance 

r0 = zeros(1, M); % linear model coefficients (continuous 
variable) 

dL c meme (al, ng % linear model coefficients (continuous 
variable) 

vE = eras ll, Mp % sensor noise variance (continuous variable) 


pe jasa = m9 ll, 1) y 
v_pred = zeros(1, K); 


pe woela = Am (al, 1) y 
y owoce = emos (il, 19) 7 


x smth = zeros(1, K); 
NN amda = os ll, 19) 7 


p updt = zeros(1, K); 


base prob - sum(n) / length(n); 
tol = 1e-8; % convergence criteria 


A - zeros(1, K); 
W = zeros(1, K); 
K) ; 
K); 


1 


CW = zeros(1, 
EX eros 


ve(1) = 0.005; 
x smtb(1) = 0; 
ZOL = 0.0037 
TLD = 0001F 
vr(1) = 0.002; 
b0 = log(base prob / (1 - base prob)); 


if (k == 1) 
x pred(k) = x smth (r1); 
v_pred(k) = ve(m) + ve(m); 
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x pred(k) = x updt(k - 1); 
v pred(k) = v updt(k - 1) + ve(m); 
end 
x updt(k) - get posterior mode(x pred(k), v pred(k), r(k) 
3 O.) el Gn), Sae (i) p. ta, AO 
p updt(k) = 1 / (1 + exp((-1) * (bO + x updt(k)))); 
be (ums) == 0) 
v updt(k) = 1 / ((1 / v pred(k)) + p updt(k) + (1 - 
p_updt (k))); 
elseif (n(k) == 1) 
v updt(k) = 1 / ((1 / v pred(k)) + ((ri(m) ^ 2) / vr( 
m)) + p updt(k) « (1 - p updt(k))); 
end 
end 
x smth(K) = x updt(K); 
v_smth(K) = v updt(K); 
W(K) = v smth(K) + (x smth(K) ^ 2); 
A(iz (end = 1)) = viupdt (1: (end T vi pred'(2:end); 
for E = UK = Dyed) 71 
x smth(k) = x updt(k) + A(k) + (x_smth(k + 1) - x pred(k + 
T)» 
v_smth(k) = v updt(k) + (A(k) ^ 2) « (v smth(k + 1) - 
v pred(k + 1)); 
CW(k) = A(k) * v smth(k + 1) + x smth(k) * x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 


if (m « M) 


R - get linear parameters(x smth, W, r, pt); 


rO(m + 1) = R(1, 1); 
ri(m + 1) = R(2, 1); 
vr(m + 1) = get maximum variance(r, r0(m + 1), rl(m + 1), 


W, x smth, pt); 


ve(m + 1) = (sum(W(2:end)) + sum(W(1:(end - 1))) - 2 + 
sum(CW)) / K; 


mean dev = mean (abs ( [ve ( d SO = b) Sim + 1 vola + 
1)] - [ve(m) r0(m) r1(m) vr(m)]) 
if mean_dev < tol 
arca (i) cs Echa cx aie sare) Ex Sales ral E iia 
yave O e O 3,18 f n\n ema es Ela (al), Onm ad), e 
(m), ve(m)); 
fprintf (Converged at m = %d\n\n’, m); 
break; 
else 
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print (a $dinx0 = %.18f\n\nrO = %.18f\nrl1l = $.18f 
ars cs SU ene = Bole vaa, Tu. Se Emma (Gb), 20m + 3b), seal (im 
+ 1), vr(m + 1), ve(m + 1)); 


Ge I 


se joel = meros (il, 19) E 
x joel cx maroni, 19) 7 


e toco S mero (al, 10) y 
Wiehe = zacon; 19) 7 


amen (seme!) c Hemos (al, i = 3D B $ x smth(1) 
needed for next iteration 
v amda = Laos (a, 19) 7 


PRUPAEN= eros MEKE 


Aeros TIK) 
W = zerosii, Ki; 
CW = zeros I K); 
Ga renos lll, E) 


; 


1 


end 
end 
end 
p updt = 1 ./ (1 + exp((-1) + (bO + x updt))) ; 
sua ex db ./ (GL 4 es) s» (90 = ss Sml))) 5 
r smth = r0(m) + rl(m) » x smth; 


lo: cx MEN ey caes (il) 19) 7 
r_plot (pt) = r(pt); 


figure; 

subplot (411); 

hold on; 

Sitemap i ‘color’, "bi, markergrze^ 4); 
plot (r_ d ce atea, al oo) 

ylabel( a leh ie seg) 

Erretes b with Simulated Data’); 

gaidi 


subplot (412); 


hold on; 

plet(p "des 

plot (prsmeh rc meat E 
ylabel('(b) p_{k}’); 

grid; 


subplot (413); 


held on; 

pilotlar ba) 

plot(x es Desire wall a) 
ylabel (' eR E ); 


xlabel (' ane index’ ) ; 
grid; 
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subplot (414) ; 

qqplot (x - x smth); 

title(’QQ Plot - State Estimate’, 'FontWeight', 'Normal'); 
ylabel('(d) input quantiles’); 

xlabel(’standard normal quantiles’); 


eod 

vacia y = ESE pos eni onlmoecde esp reci INI pred vay se), sel, We, 
D, nj 
M - 100; % maximum iterations 
y = NaN; 


end 


alie. ex Gu, IM) 
£ = zeros (al, eM) 
cha = Menos (il, M) 


sb (a) = x ecl; 
for 3.2 TEM X 
she {Ga == O) 
C = v_pred; 
(1) c ate (a) = se 9L = (G wx (im = Exodo + aka / (a 
+ exp(bO + it(i)))); 
df(i) = 1+ C « exp(bO + it(i)) / (1 + exp(bO + it(i) 
)) * 2; 
elseif (n == 1) 
C = v pred / ((r1 ^ 2) * v pred + vr); 
Et) => (a) = jc = G o (uw (ex e x s 
x pred) + vr x (n - (1 / (1 + exp((-1) + (b0 + it(1)))))); 
df (i) = 1+ C x vr x exp(b0 + it(i)) / ((1 + exp(bO + 
ig) ^ 25 
end 
aie (GL s 3) = ste (al) = Ela) / Che (al) A 
she close (6 ss al) = shie((at)))) ca deua 
Ww = dekli $ il) 
return 
end 
end 


error (‘Newton-Raphson failed to converge.'); 


function y = get maximum variance(z, r0, rl, W, x smth, pt) 


x smth - x smth(pt); 
W = W(pt); 

Z z (pt); 

K = length(pt); 


Il 
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vela su es (560 > 2) = Gal > By SS ño 
= 2 gs se) c» sun) = 2 ors mL: clee(< smea, Ta) cp AS 
TOL Sun (Rs men) KR; 


end 
function y = get linear parameters (x smth, W, z, pt) 


smth = x smth(pt); 
W(pt) ; 

z(pt); 

length (pt); 


X 
W 
z 
K 


= [K sum(x_smth); sum(x smth) sum(W)] \ [sum(z); sum(z .x 
. emth)] ; 


Mw 


end 
10.5.2 Experimental Data Example 


load('expm data one mpp.mat'); 


K 
n 


length(u); 
zerom(T. Kj 


[i 


Il 


joe = ibas > (9) 


iq) e di 

ie = wp 

M = Det 

ve = zeros(1, M); $ process noise variance 

r0 = zeros(1, M); % linear model coefficients (continuous 
variable) 

db c eros (al, Mg % linear model coefficients (continuous 
variable) 

AS cx Persas, M) y % sensor noise variance (continuous variable) 


Pe gasa cx amos (al, 9) 7 
v_pred = zeros(1, K); 


x updt = zeros(1, K); 
m woche = cu (Gal, 19) y 


x smth = zeros(1, K); 
v_smth = zeros(1, K); 


pEUDdUE-Eeros t MER 


base prob - sum(n) / length(n); 
tol = 1e-8; % convergence criteria 


A 
W 


uo dg 
N N 
0 0 
BR 
O O 
n ua 
ee 
Bs 


182 10 MATLAB Code Examples 


CW zeros KE 
Ce vacua Nj 


Wei) = 0,0057 

x smth(1) = 0; 

ZOG = 0,0037 

ZL = 0.001) 

bai = 0,0025 
( 


b0 = log(base_prob / (1 - base_prob)); 


ton m=) uM 


if (k == 1) 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
esa) = sx_ibjocle (ie = al) 9 
v_pred(k) = v_updt(k - 1) + ve(m); 
end 
x updt(k) = get posterior mode(x pred(k), v_pred(k), r(k) 
5 SeO (un), O), was) , 190, ua) 2 
plLupdt(k) = 1 / (i + exp((=1) (BO + x pdt (e) 
if (n(k) == 0) 
VAUPINIS) M P AN C D AM M ox aT 02) 0 Net pr upade 07) E (lane 
p updt (k))); 
elseif (n(k) -- 1) 
y wiee) = 1 / (4 / y pull) s (Ge) ^ 2) / val 


m)) + p updt(k) « (1 - p_updt(k))); 
end 
end 


x smth(K) = x updt(K); 
v smth(K) - v updt(K); 


W(K) = v smth(K) + (x smth(K) ^ 2); 
Ac: (end => 1)) = yv updt (a: (end! - 1)) ./ viopredi(2zend) ; 
for ko = UK = li dl 
x smth(k) = x updt(k) + A(k) » (x smth(k + 1) - x pred (K + 
po = v updt(k) + (A(k) ^ 2) « (v smth(k + 1) - 


vi predi(k +10); 
CW(k) = A(k) * v smth(k + 1) + x smth(k) * x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 

II) 


R = get linear parameters (x_smth, W, r, pt); 
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aie (a, dy > 0 


ro (m + 1) = R(t, 1); 
clin + 1) = NB 2); 
vr(m + 1) = get maximum variance(r, rO(m + 1), r1(m + 


i), Wy sx miwiin, je) y 
else $ a check with experimental data (in case this 


happens) 
fprintt (An cb = se USE nro Ma TCU c Tef 
aa O ce O Esa tmn me es Sua (6D), Om) ca (Ga), S; 
(m), ve(m)); 
fprintf (' Converged at m = %d\n\n’, m); 
break; 
end 
ve(m + 1) = (sum(W(2:end)) + sum(W(1: (end - 1))) - 2 + 


sum(CW)) / K; 


mean_dev = mean(abs([ve(m + 
) 


( qo cem + Ob) im ea) ee ou 
1)] - [ve(m) rO(m) ri(m) vr(m)]) 


YE 


if mean dev < tol 
fprint (Uu c CEN tc Ey ao rs ase iow cs E Aa 
das ca SUN cc Bole a, i, we fiui ub) , 20m), seal (im), we 


(m), ve(m); 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
fora Ex (mm CO webs) = s LS oaks) ex e SMe al Eo ashe 


Nova = g. Sit \aves =) 3. datnr m, x smeni), rom k D rim 
+ 1), velm + 1), vel 


x pred = zeros(1, K); 
y ise e maroni, 19) 7 


e ose e marmo OPEKE 
pdt zeros MF 


pomi eremo = rosal, E M $ x smth(1) 
needed for next iteration 
smt he T eros (al, 1) y 


Jo wole c magos (a, HQ E 


A = werosg(i. Ki; 
W s xzerosti. Ki; 
KF 

KIE 


£ 


CW = zeros(1, 
C= Zero, 
end 
end 
end 


p_smth 
r_smth 


43 ./ (X + exp((-1) + (bO + x smth))); 
rO(m) + rl(m) + x smth; 


Il 
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ais > momia 025, 29 a Seras ly emt ope; 

ucl x = norminv(0.975, x smth, sqrt (v_smth)); 

certainty = 1 - normcdf(prctile(x smth, 50) x ones(1, length( 
x smth)), x smth, sqrt(v smth)); 

piel js» cx G hy 1) y 

pyedl jo) Zeros (al, 10) 7 

for k = 13% 
lei vel js) || = ger jas Gom as (Y Eme (is), 190, ¿e Sucia 
(Q9) p 

end 

fo = 4 

Se (Ose = 3) / ue 

tr = ((K - 1):(-1):0) / £s; 

u plot = NaN » ones(1, K); 

u plot (pt) ale) A 

ERG be (SIHF 

hold on; 

polo (ew, Wy, “ie, GT , bo) p 

ylabel(('(a) skin cond.’ aa E 

set (geca, xtieklabel Ayma (oy Spi 

title(’State Estimation with Experimental Data’); xlim([0 ty(end) 
DE 

grid; 

Nall = also 

parten (xpi) xp (2) PES AA 
Mo shia racehlpha*, 0.2, '"EdgeColor^, none”); 

parch baala), A sso(59 5 Y 4 xo Ib) will) yl) sat 
Mi es racehlpha*, 0:2, EdgeCcolor ^ mone) 

patch(Ixp(3), xp(4), xp(4), xp(3)] / fs, [yl(1) y1(1) y1(2) yl(2 
], [1 0.647059 un 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

paran Era), 5495) 5 a), sse Y ses, AA) wall) sub» ae 
lc Us. haoc 0.2, Edgecolor/ “nonet; 

patch(Ixp(5), xp(6), xp(6), xp(5)] / £s, [yl(1) yl(1) yl(2) y1(2 
ea: jore a 02 Edge color ee none); 

subplot (512 

O u plot. UEIIT!, kh’, ^markersize', 3); 

ylabel( zy Web. e (ep) m crauclp salia (O elec) slam (© 151), 

yl - me e n ar rok labea dS 

¡arca solih; o), ss9(2))5 06 7 ES. NL) LA) e sab 
Io wen 'racenlpha^, 0.2, 'EdgeColor', 'none'); 

¡bc ba, sol), oa), se(2)1] Y Bs, Mi) LE) ye) seb 
| ere racenlpha^, 02, wdgecolor ^ none!) 

¡arc ESE), 39 (629) 5. ssa), ME 7 ies, [byl (ab) A) yaa) ye) 
lo I3 0.547059 a 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 
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¡aran (Ea), a(S), E), aay Y ws, AA) wal) wily vale) 


) 
I bite Baca pra 0:2, Bdgecolloc/ "none 
¡arca sso), a36); S96), MS Y es, (yall) AA) wala) se 
IAB aca pra 0:2, Edge colon , none); 
subplot (513); 
hold on; 
pillaje (e, Se Sinica, Yl’, Clases) ab 25) 7 
EaU (le, desa y Hel sx Flip (mel $<) I), E, xke”, “areas, Y 


FaceAlpha', 0.5); 
ylabel (' (c) state (x_{k}) 
set (gca, 'xticklabel', []); xlim([0 t(end)]); 
guanoa aL = silet 


parc bas (il), ae), sq), sso(3) Y see, [byl (GD) seb (GU) T wb 
Ie ass ‘racentpha 01:25 “EdgeColor’, “nonet; 

pArA) MS), ss9(50 5 MA 7 xs (lab) sib (al) wall) sala 
lA e 'sacenlpha^, 0.2, 'EdgeColor', 'none'); 

aca, Seo), 5:912) 5». 391591] 7 tei, AE) sal (Gk) wale) vale 
ly [2 03647059 a, 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

patch(Ixp(4), xp(5), xp(5), xp(4)] / £s, [yl(1) yl(1) yl(2) yl(2 
pe to (racehlpha*, 02, Edge color | nons. 

[ocu En), 249(6), 3908), an) / Ese, wd) y) ya) yv (a 
Nie ee pia ee 0.2, 'EdgeColor', ‘none’); 

subplot (514); 

hold on; 

plott pismen 4322, siesta SDN 

Edi (les el, Bela Eltol(mel jo), ll, O, (3022 / 255) 11, ^ 
EdgeColor’, ‘none’, 'FaceAlpha', 0.3); 


iyaltatm MO ©, a5) a 

ylabel('(d) probability (p (k]))') 
set(gca,'xticklabel', []); xlim([0 t(end)]); 
pasla UL = sib 


parecia Esa), sx9 (20); (2), ss9(30)1 a) bab) sy (al) atq) sab 
H aci L 0m2 S S Te L H R Te LE 

¡arca Ese), a), ae), MB) Y Es, IAE) H sabe» sue 
Nps egies, racehlpha*, 02 Bdgecolor 7 none”); 

patch([xp(3), xp(4), xp(4), xp(3)] / £s, ly1(1) yl(1) yl(2) y1(2 
Jo (i 0-647059 a 'FaceAlpha', 0.2, 'EdgeColor', 'none'); 

parc (ra), (8), sso), sse) Y tee, ELA wall) se» ae 
TI cts b E 0.2, 'EdgeColor', 'none'); 

peccal taala); e605), T sae) il e IAE) sab (Gb) yal) yaa 
|] yn a aa 02, “Edgecolor!, nonet); 

subplot (515); 


hold on; 

vl = [0 079. Elena), 0.9. Elena) 1: 0 Ll 

ci = n (220 / 255) (220 / 255) 5 1 (220 / 255) (220 / 285), 1 © 
Oe L o Ols 

facesl = [1 2 3 4]; 
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patch('Faces', facesl, 'Vertices', vl, 'FaceVertexCData', cl, ' 
BaceColor/;, “interp, 3 
'EdgeColor', 'none', 'FaceAlpha', 0.7); 


ll 


v2 [0 wp tend) 05 Elena! 0-T 0 0.115 

G2 e (© 0,38 07 0 Us OF (204 / 255) 1 (204 7 255). (204 / 255) 3 
(204 / 255) ils 

faces? = [21 2 3 4]; 


patch('Faces', faces2, 'Vertices', v2, 'FaceVertexCData', c2, ' 


FaceColor', 'interp', x 
'EdgeColor', 'none', 'FaceAlpha', 0.7); 
[ose (e. certakaey, "ess, (ass / 255) (43 / 255) (226 / 255) 1, 
t mewddth4 an5); grua; 
ylabel('(d) HAI’); xlabel('time (s)'); xlim([0 t(end)]); 
tunctdonel-sgetgposterriornocdeoepred/vpred mz Orr vas, 
D, m) 
MESO % maximum iterations 
alice ex excusas hug 
i8 c3 zeros (PEM) E 


df - zeros(1, M); 


mes ab e g/l = 11) 
alae (a Es 0) 
C = v_pred; 
f(i) = it(i) - x pred - C « (n - exp(bO + it(i)) / (1 
+ exp(b0 + it(i)))); 
df(i) = 1+ C * exp(bO + it(i)) / (1 + exp(bO + it(i) 
)) ^ Be 
elseif (n -- 1) 
€ = Wissel Y (Gal ~ 2) re ve); 
Est) c te (al) = med = € o (elo (a = x0 = los 
sx joc!) «e e a (as (GU / Gl de sed) & (oo + ately ye 
df(i) = 1+ C « vr « exp(b0 + it(i)) / ((1 + exp(bO + 
aeu» * 255 
end 
ae (ak gb 1) a SEQ) = 26) / GRE (al) E 
if abs(it(i + 1) - it(i)) < le-14 
y= it(i + 1); 
return 
end 
end 


error('Newton-Raphson failed to converge.'); 
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end 
function y = get maximum variance(z, r0, r1, W, x smth, pt) 


o smt exe Smth pt) 


W = W(pt); 
n ex OIE) $ 
K = length(pt); 
ye (2 xs ox sexso (60 7 2) c Gal ^ EX e sunlit) 
* r0 « sum(z) - 2 « rt » dot(x smth, z) + 2 * 


= 2 
r0 * rl «x sum(x smth)) / K; 
end 


function y - get linear parameters(x smth, W, z, pt) 


x smth - x smth(pt); 
W - W(pt); 
Z 
K 


ZI (pe) 
= length(pt); 


= [K sum(x smth); sum(x smth) sum(W)] \ [sum(z); sum(z .x 
. emth)] ; 


* o 


end 
aite Hel, ve = ger js cone aims, 190, 9) 
i) e (ue-asieedsi) s 
15s) ecumterapz (pr de cJ (clencie (2 al es at) 22 ys) oe (Gl = 192) 9) oe 
exp(((-1) / (2 * v))* (1og(p ./ ((1- p) * exp(b0))) - x) 
n - find(fp «- 0.975); 
me alas <= 04023) + 
ucl = pín(end)); 


D 
lel = p(m(end)); 
end 
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10.6 State-space Model with One MPP and One Continuous 
Observation 


10.6.1 Simulated Data Example 


load('data one mpp one cont.mat'); 


base prob - sum(n) / length(n); 
joie e kaelih => 0) y 


$$ parameters 


M = 1e6; $ maximum iterations 

lun = ; 

tol = 1e-8; $ convergence criteria 

rO = xeros(i, Mi); % continuous model 

EOS (CAM 

vr = zerosíl, My % continuous model noise variance (1) 
S0 = zeros (1, M); % continuous model 

EL. = mes, If 

vg = zeros(1, M); % continuous model noise variance (2) 
ve = zeros(1, M); $ process noise variance 


|o wüselz c eene, 1) 


A - zeros(1, K); 
W = xeros(i. Kl; 
KOR 
K); 


Z 


CW = zeros(1, 
C= zeros 1, 


$$ initial guesses 


b0 = log (base prob / (1 - base prob)); 

r0(1) = r(1); $ guess it's the first value of r 
az db (aL) = 0555 

SO (1) SEDE 

s1(1) = 1; 

vel = 0.05; 
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ae (Ue == iL) 
x pred(k) = x smth(1); 
v pred(k) = ve(m) + ve(m); 
else 
x pred(k) = x upde(k = 195 
v_pred(k) = v_updt(k - 1) + ve(m); 
end 
x updt(k) = get posterior mode(x pred(k), v pred(k), r(k) 
7 LOU), seal), 190, spe (un) mus), SCs), cu); cuba); ws) y 
p_updt(k) = 1 / (1 + exp((-1) + (DO + x updt(k)))); 
de Gal (Re) == 0) 
v updt(k) = 1 / ((1 / v pred(k)) + ((si(m) ^ 2) / vs( 
m)) + p updt(k) s (1 - p_updt(k))); 
elseif (n(k) == 1) 
y mas) = 1/ (1 / y arde) e (Elm ^ 2) / vel 
m)) + ((s1(m) ^ 2) / vs(m)) + p updt(k) * (1 - p_updt(k))); 
end 
end 
x smth(K) = x updt (K); 
v_smth(K) = v updt(K); 
W(K) = v_smth(K) + (x smth(K) ^ 2); 
A(1: (end - 1)) = v_updt (1: (end - 1)) ./ v_pred(2:end) ; 
for k= UK = yr 
x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - x pred(k + 
10)) f 
v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 


v pred(k + 1)); 


CW(k) = A(k) « v smth(k + 1) + x smth(k) « x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 

end 

prev = [rO(m) ri(m) ve(m) vr(m) s0(m) si(m) vs(m)]; 


R - get linear parameters for mpp(x smth, W, r, pt); 
S = get linear parameters(x smth, W, s, K); 


ve(m + 1) = (sum(W(2:end)) + sum(W(1:(end - 1))) - 2 * sum(CW 
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seo (np 1) c ial, are 

nail (im se 3D) e IRC, 1)5 

SOG + 1) = SG, a); 

alma 1) = SE 3g 

vr(m + 1) = get maximum variance for mpp(r, rO(m + 1), r1(m + 
1), W, x smth, pt); 

vs(m + 1) = get maximum variance(s, sO(m + 1), sl(m+ 1), W, 

x_smth, K); 

next = baoa e 1) cl(m + 1) veim - 1) velm + 1) som + 1) sul 

m+ 1) vs(m +1)]; 

mean dev - mean(abs(next - prev)); 


if mean dev « tol 
fprintf('Converged at m = #d\n\n’, m); 


break; 
else 
orcas (Mn = xeon) cs Sena ens oie waite E dba Ma Mele) 
= %.18f£\nsl = %.18f\nvs = %.18f\n\nve = %.18£\n\n’, 
me 1, oum 1), clima = 1). vrim « 1). sim - 1), slim e 
i), sem c 3) vela + 10): 
x pred = zeros(1, K); 
v pred - zeros(1, K); 
sosa Meroe MERE 
y oso = mios ll, 19), 
se ama (Beca) c mozos (il, Ke = 300p % x emen (1) needed 
for next iteration 
v smth = zeros(1, K); 
i9 Waste c zeros PRE 
Pu EA: E 
W - zeros(1, K); 
CW - zeros(1, K); 
Ce rue. 19 
end 
end 
$$ calculate confidence limits 
p smth = 1 ./ (1 + exp((-1) x (bO + x smth))); % mode, lower and 
upper confidence limits for binary distribution 
r smth = r0(m) + rl(m) » x smth; 
s smth = s0(m) + s1(m) + x smth; 
r plot = NaN x ones(1, K); 


r plot (pt) 


2 (De) $ 
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%% plot graphs 


subplot (511); 


holdkon; 
Seco las, Aela “ele, Uat, E E , 0) y 
plojs (se Sala, "= a Ciela! , 129) p 


E TE 
title(’Estimation with Simulated Data’); 
grid; 


subplot (512); 


hold on; 

pileta, US), 

plc fa smda, "s." Clica, 1425) p 
ylabel('(b) p_([k)'); 

guta 


subplot (513); 


hola on; 
lore, 190) 
lo TT (r u inewidth aL 25), grid; 


ylabel('(c) s_{k}'); 


subplot (514); 


hold on; 
pilots cc “bir 
jolie (be emay ss", Mihaela 1.25) Cale 


ylabel('(d) x (k)'); xlabel('time index’); 


subplot (515); 

qqplot (x - x smth); 

title(’QQ Plot - State Estimate’, 'FontWeight', 'Normal'); 
ylabel('(e) input quantiles'); 

xlabel('standard normal quantiles'); 

grid; 


$$ supplementary functions 


Function vE-Egetlposterionamodex opi ecoViprecdoee sel), seal, Jeo), 
7 thy By SO, SL, ws) 


M = 200; % maximum iterations 
te = eso, Il) 

i8 cx ero, pls 

cha. = Eros (Ll, M) 

she (a) = sm gere 


for i = 1:(M - 1) 


if (n == 0) 
C = v pred / ((sl ^ 2) « v pred + vs); 
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Ea) => ake (al) = jc) = € e (Sil se (E = 0 = Sil s 
x pred) + vs « (n - (1 / (1 + exp((-1) + (bO + it(1)))))); 
df (i) = 1 + C « vs x exp(b0 + it(i)) / ((1 + exp(bO + 


ac») ^ 2) 5 
elsert oin == 1 
C = v pred / (vr « vs + v pred « ((rl ^ 2) « vs + (sl 
^7 8) e ws 
(31) = ae) = ox jormecl = € 6 (Gu e ws e (G9 = mil i 


E 
pre SS AS OS A e a 
vr x vs * (n - (1 / (1 + exp((-1) » (bO + it(i))) 
DEDE: 
df (i) = 1 + C x vr x vs + exp(b0 + it(i)) / ((1 + exp 
190 > sequ) ^ 295 
end 
aie (aL s 3D) e dela) = 2) / Gh (al) yg 
she clos alea ss 30) = siie((at))) m emi 
s; e dela y al) 5 
return; 
end 


end 
error('Newton-Raphson failed to converge.'); 
end 


function y = get maximum variance(z, r0, rl, W, x smth, K) 


y= (zx z! +K» (r0 2) a (ass ^ 2) & eun) 
= 2 xs we) c» Sun) = 2 ors sel c» (le(s murus Ta) FAS 
r0 * rl * sum(x smth)) / K; 


end 
function y - get linear parameters(x smth, W, z, K) 


y = [K sum(x smth); sum(x smth) sum(W)] \ [sum(z); sum(z .x 
x smth)]; 


end 


function y = get maximum variance for mpp(z, r0, rl, W, x smth, 
pt) 


x smth - x smth(pt); 


m ee veu) $ 
K = length(pt); 


y= LZ xS + K (550 ^ 95. s Gal ^ B) c9 Ema 
- 2 r0 « sum(z) - 2 « ri « dot(x smth, z) + 2 * 
20 5 sl o suas suda) ER 


* 
* 


end 
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function y = get linear parameters for mpp(x smth, W, z, pt) 


x smth - x smth(pt); 
W = W(pt); 

Z z(pt); 

K = length(pt); 


y = [K sum(x smth); sum(x smth) sum(W)] NV [sum(z); sum(z 
x smth)]; 


end 


10.6.2 Experimental Data Example 


load('expm data one mpp one cont.mat'); 


Silos) = Iz 
T = 1450; 
n = zeros(1, T); 
C= zeroed d Te 


joe = TT zx (0) 


n(pt) = 1; 
r(pt) = u(pt); 
e == 


base prob = sum(n) / length(n); 
¡fe = eein S> 10) 7 


$$ parameters 


M = 1e6; % maximum iterations 

n = Als 

tol = 1e-8; % convergence criteria 

bO = zeros(1, M); 

bl = zeros(1, M); 

rO = zeros(1, M); % continuous model 

rl = zeros(1, M); 

vr = zeros(1, M); % continuous model noise variance (1) 
S0 = zeros(1, M); % continuous model 

sl = zeros (1, M); 

ve = zeros(1, M); % continuous model noise variance (2) 
ve = zeros(1, M); $ process noise variance 


.* 
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pe ace = amo, N95 
sele = amos E MERE 


Pe amda c» 220 (Gl, Ws 
HZ smell = uo (Cal, 19) y 


p updt = zeros(1, K); 


A 
W 
CW 
© 


o KG 
, K); 
= zeros(1, K); 
) 


zeros (1 
zeros (1 


zeros(1, K); 


initial guesses 


log(base prob / (1 - base prob)); 


) 

) = aly 

b) = eerie loc), DO) 7 

|o = 057 

) e alii; 

) = 1; 

D = 0.05; 

Ed var(s); % 1 x var(s) 
) e Ds 

d. 


alae e exo 3b) 

x pred(k) - x smth(1); 

v pred(k) = ve(m) + ve(m) ; 
else 


x pred(k) - x updt(k - 1); 
v pred(k) = v updt(k - 1) + ve(m); 


x updt(k) - get posterior mode(x pred(k), v pred(k), r(k) 
7 OC), seal (ay), lim), lok), uae un) n ats), SU, SO), calm) 


p updt(k) = 1 / (1 + exp((-1) « (bO(m) + b1(m) « x_updt(k 
DOD < 
e (als) ee 0) 
v updt (k) / 1 / v pred(k)) + ((s1(m) ^ 2) / vs( 
= 2) 


k 
m)) + (b1(m) 2 
elseif (n(k) 
A (k / ((1 / v_pred(k)) + ((r1(m) ^ 2) / vr( 
m)) + ((sl(m) ^ 2) / vs(m)) + (b1(m) ^ 2) « p updt(k) * (1 - 


p updt (k))); 
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end 

end 
x smth(K) = x updt(K); 
y smc (19) c we quel (1) y 
W(K) = v smth(K) + (x smth(K) ^ 2); 
A(1: (end - 1)) = v_updt (1: (end - 1)) ./ v pred(2:end) ; 
for k= (kK = Mid IL 

x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - pred (K + 

3L) y p 
v_smth(k) = v updt(k) + (A(k) ^ 2) « (v smth(k + 1) - 


v pred(k + 1)); 


CW(k) = A(k) * v smth(k + 1) + x smth(k) + x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 

end 

R - get linear parameters for mpp(x smth, W, r, pt); 


S - get linear parameters(x smth, W, s, K); 


prev = [rO(m) r1(m) ve(m) vr(m) sO(m) si(m) vs(m) bO(m) bi (m) 
l8 

ve(m + 1) = (sum(W(2:end)) + sum(W(1: (end - 1))) - 2 + sum(CW 
DI f ie 


bb = fsolve(e 


(b) binary parameter derivatives(b, n, x smth, 
v smth), [-5 1] 


, optimset('Display','off')); 
ley) (um se 2D) = deje E 
¡sal (a + 11) => llo (a) + 
Som + 1) a 861. 195 
Slim + 1j m Bla, Ay 
vr(m + 1) = get maximum variance for mpp(r, rO(m + 1), rl(m + 
i), Wi E ening 19) y 


if ((vs(m) + lambda + (get maximum variance(s, s0(m), s1(m), 


Wy Bx musa 50) = sw) > 0,75 + vera) % EM algorithm 
intentionally modified slightly for overfitting control 

sO(m + 1) = s0(m) + lambda « (S(1, 1) - sO(m)); 

sl(m + 1) = s1(m) + lambda + (S(2, 1) - s1(m)); 

vs(m + 1) = vs(m) + lambda « (get maximum variance(s, sO( 
ia) SL, Mp Ema, M KS ME Mv dm P 
else 

sO(m + 1) = sO(m); 

Eu = 3D) 3 Sal (ai) s 

vs(m + 1) = vs(m); 
end 
next = [rO(m - 1) ri(m + 1) velm + 1) velm « 1) solo + 1) stí 


m + 1) vs(m + 1) bO(m + 1) blm + 1)1]; 
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mean dev = mean(abs(next - prev)); 


ie (deuL(wd em 3D) = ©) ||| Caim = o3) 2 0) % if this happens 
with experimental data 

fprintf('Iterations halted at m = %d\n\n’, m); 

break; 
end 


if mean_dev < tol 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
apra (tud cx Sylow) es Ex alee yal = Bike Ventas iaa 
= %.18f\ns1l = %.18f\nvs = %.18f\n\nbO = %.18f\nb1 = %.18f\n\ 
nve = %.18f\n\n’, 
mo 1, £roO(m + 1), riim = 1), vela + 1^ s0im + 1), slim + 
1), wein 1), bo(m * 1), bi(m + 1), velm = 1)); 


reel c» meros, 18) $ 
v_pred = zeros(1, K); 


se Werle c zeros PERKE 
v updt = zeros(1, K); 


se fimile rend) c mol, ke e al) p $ x smth(1) needed 
for next iteration 
v smth = zeros(1, K); 


D oce = zeres, EK) 


A = zeros(1, K); 
We ramal, 10) 7 
K); 
K); 


iy 


CW = zeros(1, 
C= seros(l, 
end 


$$ calculate confidence limits 


p smth = 1 ./ (1 + exp((-1) + (bO(m) + bl(m) + x smth))); % mode 
, lower and upper confidence limits for binary distribution 

r smth = r0(m) + rl(m) * x smth; 

Ss smth = sO(m) + sl(m) + x smth; 


ci noO 0025 ssmthys sque(varsmth))) a 
Cel e > eraka OS, 2 mica, exe aeic (ie DET 5 


ell js» ex Meal, p 
Cel je cx meros (abs Oy 


tom k= qm 
[1cl p(k), ucl p(k)] = get_pk_conf_lims(v_smth(k), bO(m), b1( 
Tem bs) 
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end 


E olos = WEN e CRES ER 
r plot(pt) = r(pt); 


$$ plot graphs 


pos Ae 

tr = (K:(-1):1); 

tC G k (4k60): 14507 

xtick labels = ('9 AM’, ‘1 PM’, '5 PM’, '9 PM’, '1 AM’, '5 AM’, ' 
9 AM'); 


subplot (411); 


hold our 

GH pot MESE EE color MES MT. ense 4) y 

[olo (E, x Mel, "seg". Alne a 3L.25) p 

ylabel('(a) n (k), r (k]'); ylim([-inf (max([r_ plot, r smth]) + 
2 55) e 


grid x emaor “slg ser ejes, Vedas”, kei ck iposi 
set(gca, 'xticklabel', []); 
title('State Estimation with Experimental Data'); 


subplot (412); 


hold on; 

local, E) "eux, BL (12 / 2585) 0, basaren 1.25) cgo 
pokexe (e, E Ema, “>, a nevi dth , EI 

ylim([O (max([s, s_smth]) + 2.5)1); 

ylabel('(b) s (k)'); set(gca, 'xtick', xtick pos); 


Palak, KIN eSeleeey, e xt ella e [HL 


subplot (413); 

bold on; 

col = [0 (176 / 255) (80 / 255)]; 

cada le, el, Mel jo clap (fuel 19) ] 5 (Sse / 255) (208 / 255) (80 
/ 255), *BdgeCollor’, “mone, “FaceAlpha’ 03); 

loe, i19 Ema, "cer", (654 / 255) G50 / 255) (80 / ASS), " 
linewidth’, 1.25); grid; 

ylabel('(c) p (k)'); set(gca, 'xtick', xtick pos); ylim([0 (max( 
ucl p) + 0.0075)]); 

ESD lO, Ise ES lees, E RT E S ID) 


subplot (414) ; 


held on; 

al (le, EX], dled se iole e s) 1, E L / 255) O (204 / 255) 1, 
'EdgeColor', ‘none’, 'FaceAlpha', 0.3); 

loja (ie, 22 Ella, “color”, [Oe / 255) © (aum / 255), “lsbasiiloldal 
, 1.25); 

[speciei »elbstwa( (MO, A) a ca e >) = 3) (meme >) > aD) I))) 5 


eco (Geel, “abuela, zela (998) y 
setisgcar abu clabel p. enki disllexsilie)) p 
ylabel('(d) x (k)'); xlabel('time'); 
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PUNEO y > Get Posterior Wole la oeae, v orech, 1, se), sail, D0, i 
y Wey, m By SO, Sil, VE) 


M = 200; % maximum iterations 
SIS SOS hup 


Zero Ss (AM) 
df = zeros(1, M); 


p 
et 
un 
Il 


x pred; 


for i = 1:(M - 1) 


"E pec 
C = v pred / ((sl ^ 2) * v pred + vs); 
ie (al) c ate (al) = ox jewel = € o (mu e (m = S = sil y 
x pred) + vs « bl » (n - (1 / (1 + exp((-1) * (bO + bl x* it(i 
DIM) 
df(i) = 1+ C « vs « (bl ^ 2) + exp(b0 + bl * it(i)) 
PUE 2 eno s feu e de Gi) ^ 2 
elseif (n == 1) 
C = v pred / (vr * vs + v pred « ((rl ^ 2) * vs + (s1 
^ 2) c wy 
EU = de (al) = x jewel e € a (suh o ws o (ie = rO = il s 
Se) x fel v» was c (ep oe 0 > Sil c sx exe) + 
vr x vs * bl » (n- (1/ (1 exp((-1) + (bO + bl 
& she QD)» 
df(i) 2 1 + C « vr « vs « (bl ^ 2) « exp(b0 + bl x it 
GD) Y UA s esos) « tol v» xbe(3020) ^ 295 
end 
leal s 35) e sey = (a) / Ghe (al) y 
if abs(it(i + 1) - it(i)) < 1e-14 
y= it(i +1); 
return; 
end 


end 
error('Newton-Raphson failed to converge.'); 
end 
HT (lent, welll] 4 ceu jos (Kex sims, 190), Jedi. 4) 
p = (1e-4:1e-4:1); 


ie. = emeen dL of (cese x qun c3) w lol ej) a (1 o 19) 


.* 


exp(((-1) / (2 * v))* ((1 / b1) * log(p ./ ((1 - p) + exp 
(590) )) = s) .^ By) s 
ih ex nea <=] 0,375) y 
messa = 0,025) F 
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quel 
lel -= 
end 


p(n 


TE 2 


pt) 
x_smth = 
W = W(pt) 
n ex akor) 
== lenge 
Ws (4&8 
EO x 


end 


function y - 


end 


function y - 


= z(pt) 
-mtengt 

 smth)]; 
end 


eEG SDS va 


y = 
x_smth)]; 


end 


Funct loniy = 


y = zeros 
K = lengt 
SO cx Te ab) 
bl = b(2) 
DE-EZenunos 


(exero) p 


p(m(end)); 


get maximum variance for mpp(z, r0, 


x smth(pt); 


1 
D 


BE 


2 ERK 
= 2 
( 


* sum(x_smth) ) 


get_maximum_variance(z, 


Zee 
= 2 
( 


* sum(x_smth) ) 


* 
* 


* 
* 


(50 ^ 2» = 
ro * sum(z) 
if Hg 


(sai, ^ 2) 


TOS 


(sex * 2) 


* sum(W) 
- 2 * rl x dot(x smth, 


x sum(W) 
= 2 o veil xs cles men, 


get linear parameters for mpp(x smth, W, 


x smth(pt); 


1 


h (pt); 


[K sum(x smth); sum(x smth) 


sum (W) ] 


\ 


get linear parameters(x smth, W, 


[K sum(x smth); sum(x smth) 


sum (W) ] 


\ 


Z, 


[sum(z) ; 


K) 


[sum(z) ; 


z) 


iL, Wy x Simtel, O) 


z) 


ZP E) 


sum LZ 


sum L Z 


199 


ido Wy 2 miele. 


+2 + 


+ 2 * 


x 


.* 


binary parameter derivatives(b, n, x smth, v smth) 


(aly 207 
h(n) ; 


1 


G IS) s 
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p(k) 1 / (1 + exp((-1) * (b0 + b1 » x smth(k)))); 

y(1) = y(1) + n(k) - p(k) - 0.5 « v smth(k) * (b1 ^ 2) « 
joe) ee (al = Te) s Wal ccm es Tex) y 

WA) = ye) a) Pe mela) smt hom SS ON 
v amado) s oul cx joie) c» (GL = joie) x (A + se Ema) cx leul ss 
(1 - 2 * p(k))); 


end 


10.7 State-space Model with One Binary and One 
Spiking-type Observation 


10.7.1 Experimental Data Example 


load('expm data one bin one spk.mat'); 


fs = 4; 
delta = 0.005; 


min_peak height = 0.1; 
min peak promn = 0.1; 
min peak dist - fs; 


Spl; 
a En; 


ph 
En 


rpeaks = s.rpeaks; 
HL = Swi, 


W = S.W; 
theta = s.theta; 
[pks, locs] = findpeaks (ph, 'MinPeakHeight', min peak height, ' 


MinPeakProminence', 
min peak promn, 'MinPeakDistance', min peak dist); 


n = zeros(1, length(ph)); 
nilocs) = 15 


Ker ele nicht iay m) 


M = 2e4; 
ve = zeros(1, M); $ process noise variance 
Be jale meus. 9 y 


vpredE-Ezeros MEHR 


Pe wigelE c remos (1, E y 
viupdi == Mera PEKI 


be smell. = Los (Cal, 19) y 
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hz fcm = m9 (al, 19) 7 
[o wle = euer, T9 


tpc = 289; % total (SCR) peak count 
tsl = 34182; % total signal length 


base prob = tpc / tsl; 
b0 = log(base prob / (1 - base prob)); 
tol = 5e-8; % convergence criteria 


A= zeros (1, K); 
W = zeros(1, K); 
ROE 
K); 


1 


CW = zeros(1, 
C = zeros 1, 


x smth(1) = 0.44201528159733; 
ve(1) = 1.24111644606324e-4; 


eta - -0.00004; 
exception counter - 0; 
ike um cx al gli 
more Ms = ab Rie 
) c e amena) y 
) = ve(m) + ve(m); 


x pred(k) = x updt(k - 1); 
v pred(k) = v updt(k - 1) + ve(m); 


o 


201 


try % numerical issues can occur due to the integrals 
[x updt(k), H2] = get posterior mode(x pred(k), C(k), 


bo, n(k), rpeaks(k, :), ul(k, :), delta, s.w(k, 
', eta); 
p updt (k) 
v updt(k) - 
p updt(k)) - H2); 
catch 
exception counter - 
x updt(k) - x pred( 
v updt(k) - v pred( 
end 


à 
iL ff 


if (mod(k, 100) == 0 
forni da e 
end 


(aL 4 es lcd) c» (190 «s bx Woste (ie) )) y 
((1 / v pred(k)) + p updt(k) * (1 


exception counter + 1; 


an 3) theta 
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se (meis, 2500) == 0) 
Eprinta (mio) e 
end 


end 


x smth(K) = x upat (K); 
v smth(K) = v updt(K); 


W(K) = v smth(K) + (x smth(K) ^ 2); 
A(1: (end - 1)) = v_updt (1: (end - 1)) ./ v pred(2:end); 
for k-z- (ij sian 
x smth(k) = x updt(k) + A(k) * (x smth(k + 1) - x_pred(k + 
1)); 


v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 
v pred(k + 1)); 


CW(k) = A(k) * v smth(k + 1) + x smth(k) * x smth(k + 1); 
W(k) = v smth(k) + (x smth(k) ^ 2); 
end 
if (m « M) 
ve(m + 1) = (sum(W(2:end)) + sum(W(1:(end - 1))) - 2 + 
sum(CW)) / K; 
mean_dev = mean(abs(ve(m + 1) - ve(m))); 


if mean_dev < tol 
fprine nime SENO cx Ey ake Nye > Sol aie’! y up 
x_smth(1), ve(m)); 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
Eprints (m=. dino = selSt\nvel SS 
ma), asm c 3) 7 


ne joel = aro (al, 19) 7 
v_pred = zeros(1, K); 


me Whole = mero (al, 19) 7 
v_updt zeros(1, K); 


¿Em (2 end) Menos al, e = al) 5 $ x smth(1) 
needed for next iteration 
y anda = Manos (al, 1%) 7 


PRUPAEN EZ SOS (ESE 


A - Zeros il, Ki; 
W = xerosii. Ki; 
KE 

K); 


f 


CW = zeros (1, 
e zeros(i, 
end 
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end 
end 
p updt = 1 ./ (1 + exp((-1) « (bO + x updt))); 
p smth = 1 ./ (1 + exp((-1) + (b0 + x smth))); 
Es (0 (6% x)» / (Es & GE 
eu c (0k = De (Si):0) / (Es c 60), 


(CEA MO inv (00257 smt hy sqrt G n i 
ucl x = norminv(0.975, x smth, sqrt (v_smth)); 


lel jo s raros (il, 19) 
Cel jo) = mos (AL, A9) p 


fom e = Igis 


203 


[1cl p(k), ucl p(k)] = get pk conf lims(v smth(k), bO, x smth 


OE 
end 


certainty = get certainty curve(v smth, b0, x smth, base prob); 


lambda - zeros(K, 50); 
necia sare > eros E50) 


mene Te c3 Signe 
ius 5] ex 1250 


w = [squeeze(s.w(k, j, :))' [eta x_smth(k)]]; 
if (f(theta', ul(k, j), w) > 1e-18) 
lambda(k, J) = fetch lambda(theta', ul(k, j), w); 
end 
ICA A Ww) 


end 
end 


lambda_start_index = find(reshape (rpeaks', 1, numel (rpeaks)), 

lambda = reshape(lambda’, 1, numel (lambda) ) ; 

get ks plot (find(reshape(rpeaks’, 1, numel (rpeaks))) » delta, 
lambda(lambda start index:end), delta, 1); 


U = gee dec lisa (Sra, ggas; Ml, celta, So mera”, 
x smth, v smth); 

11 final = sum(nansum(11)); 

mean rr - reshape(mean rr', 1, numel(mean rr)); 


me TREI IST aoee LOCS) A 
rr times = s.rpeak locs(2:end) / 60; 


Sres vilami = iba >) = 051) e T >) ss 0,1)15 

se ylin = ISRA HS (fecal, 30) = 0.05) (foseraile (asa, 99) + 0.05 
problylim = [min (lel p)) = 0.0005) (max (uclpi3 send) 0:00 
figure; 


subplot (611) ; 


1) 5 
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bold ‘on; 

lo E, Box, "cedex". [rug / 255) © (204 / 255) 1) 5 creck 

set(gca,’xticklabel’, []); 

ylabel (' (a) z (k)'); xlim([0 t(end)]); ylim([4 22]); title('State 
Estimation with Experimental Data’); 


subplot (612) ; 

n plot = NaN » ones(1, K); 

m gode (im s (9) = dip 

L mole, “isl, “cales”, Jl, 0, il), “meras, 229 B 
O ie Creare NT Er Aa do 1.281); 

set (gca, 'xticklabel', []); 

ylabel(’(b) no 1kp0); grid; 


subplot (613); 


hold on; 

plows, se Ense, 7192, lxi" y dU) p. psstelp 

set (gca,’xticklabel’, []); 

cada Ce, eal S se sola (ell Sel, “el, merece, Visor es 4 
FaceAlpha’, 0.2); 

ylabel('(c) x (k)'); xlim([0 t(end)]); ylim(state ylim); 


subplot(614); 

hold on; 

¡lc ie, 19 Ema, "cci", [Oe / 255), O, (Sil / 255)], MENEE 
^5 dlp 2S) p- E HES 

set(gca,'xticklabel', []); 

cado (|e, tl, He jo Elsa (el joy), lll, 0, (ley / 25511, ^4 


EdgeColor', ‘none’, 'FaceAlpha', 0.2); 
ylabel (' (d) p_{k}’); xlim([0 t(end)]); ylim([0.0012 0.0388]); 
Ponencias prob Das Expo | eat 
15225) p 


subplot (615); 


hold on; 

plo (Gee eses, rae, “er, “coil”, (ll, 0.5, 0525]. 
"MarkerFaceCcolor' [20557201525] 7 MarkerSuze' 72) grid; 

set(gca,'xticklabel', []); 

mu start index = round(s.rpeak locs(2) / delta); 

plot(((0:(length(mean rr(mu start index:end)) - 1)) * delta) / 
cor mesita (muls castbiimde>s ena) bU 

ylabel (' (e) rr (i)'); xlim([O t(end)]); ylim(rr ylim); 


subplot (616) ; 
hold on; 


vi [0 0.9; tíend) 0.9; t(end) Yi; 0 11; 

cl = [1 (220 / 255) (220 / 255); 1 (220 / 255) (220 / 255); 10 
Or l o Oly 

facesl = [1 2 3 4]; 


patch('Faces', facesl, 'Vertices', vl, 'FaceVertexCData', cl, ' 
FaceColor', 'interp', x 
'EdgeColor', 'none', 'FaceAlpha', 0.7); 
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v2 LO 07 (Ciel) Oy tees) Gla 0 Walp 

e2 = IN O.8 07 0 0-6 Of (204 / 255) 1 (204 / 285), (204 / 255) 1 
(204 / 255)]; 

faces2 = [1 2 3 4]; 


patch('Faces', faces2, 'Vertices', v2, 'FaceVertexCData', c2, ' 


FaceColor', 'interp', A 

'EdgeColor', ‘none’, 'FaceAlpha', 0.7); 
lote, conter, “lov, a Eev idena n disc) s eje exsistunt [L(). 12 (Cerat) 
Ee HAT) label (time (mam) N; yla TTN; 
function [y, H2] = get posterior mode(x pred, C, b0, n, rpeaks, 


RG celte, w eull, tasker Metal) 

M = 40; $ maximum iterations 
abe, = eos, Abg 

func - zeros(1, M); 
div=szerosi( M) 

sb (a) = sm 966.) 


oye ab = ale Qui = di) 


H1 zeros(1, 50); 
H2 = zeros(1, 50); 


Il 


1350) 
lecteeze kw caai Ja St ESA 


dou 91 
w 


Il 


Il 


if (f(theta, ul(j), w) > 1e-18) $ 
lambda - fetch lambda(theta, ul(j), w); 


dl dx - dlambda dx(theta, ul(j), w); 
H1(j) = dl dx x (rpeaks(j) - lambda + delta) / 
lambda; 
H2(j) = d21ambda dx2(theta, ul(j), w) * (rpeaks(j 
) - lambda « delta) / lambda - rpeaks(j) « (dldx ^2) / ( 
lambda ^ 2); 
end 
end 
Hl = sum(H1); 
H2 = sumi(H2); 
func(i) = it(i) - x pred - C « (n - exp(bO + it(i)) / (1 + 
exp(b0 + it(i))) + H1); 
GE (Gl) Slee Cem (exp (On eaters) NS (o 00 at (C9) D NS 


2 = H2); 
ae (a sm 3D) = aie (al) = cuacha) / whe (al) s 
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if abs(it(i + 1) - it(i)) < le-14 
y e duela 49 iL) ¢ 
return 
end 
end 


error('Newton-Raphson failed to converge.'); 
end 


function [y] = f(theta, t, w) 


y = sqrt(theta(end) ./ (2 « pi * (t .^ 3))) .* ... 
exp((theta(end) « ((t - mu(theta, w)) .^ 2)) ./ 
((-2) * (mu(theta, w) ^ 2) * t)); 


end 
function [y] = intf(theta, t, w) 
y = integral(@(t)f(theta, t, w), 0, t); 
end 
function [y] = mu(theta, w) 
eta = w(end - 1); 
x - w(end); 
y = theta(1) + w(1:3) + theta(2:4)' + eta x x; 
end 


function [y] - fetch lambda(theta, t, w) 


Geli S ETIE teta ley Ww) Pp 
y e iaxseR, En ww) ol (il = Ec) y 


ss Keoki > ab) $ numerical issue 


end 
function [y] - df dmu(theta, t, w) 


y = (theta (end) (kenara, vg ^ 99) lenta, E, v9) cc le 
- mu(theta, w))); 


end 
function [y] - df dx(theta, t, w) 


eta - w(end - 1); 
y = eus huis i, Vy) e£ Stay 
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end 
function [y] = intdf_dx(theta, t, w) 
y = integral(@(t)df dx(theta, t, w), 0, t); 
end 
function [y] = dlambda_dx(theta, t, w) 


cdf = intf (theta, t, w); 


wie Keske z il) $ numerical issue 
y = 0; 
else 
ye (Gl = Ed) a ele sb, E, w) + 
E (theta; tow) o antdtidss(thetal, E) W) / (W = edt 
) en Bhs 


end 
function [y] = d2f dmu2(theta, t, w) 


y = theta(end) « (df dmu(theta, t, w) .x* ((t = mu(theta, w)) 
i 


ERIE STR ST ve) © 99» + 
f (theta, t, w .* ((2 * mu(theta, w - 3 x t) / (mu(theta 
a w) ^ A) Ng 


end 
function [y] - d2f dx2(theta, t, w) 


eta - w(end - 1); 
y = Chie cla (eins, Ey Y) oo (Gee D 


end 
function [y] = intd2f dx2(theta, t, w) 
y = meca (@((ie)) cl cb aca) Ey Wi), O, E) $ 
end 
function [y] = d2l1ambda_dx2 (theta, t, w) 
va (4 = cilena cb acia, Ey YY) = (al = caer (arca, ie, W 2 
intdf dx(theta, t, w) + 
(lene Cha (elise, Ter S (ab > cae (acres, Ey YI) + 20. 
f (theta, t, w) x intd2f dx2(theta, t, w)) / ((1 - intf( 


data, El wW) ^ 2); 


end 


208 10 MATLAB Code Examples 


function [y] = get_log likelihood(eta, rpeaks, ul, delta, w_all, 
theta, x, v) 


A 
D 


length (x); 
zeros(K, 50); 


< 
I 


Por =. ak 
fou 150 
wa lecresza y culal (e, 35 20) erca 4311 5 


if (f(theta, ul(k, j), w) > le-18) 


lambda = fetch lambda (theta, ul(k, j), w); 
dl dx = dlambda dx(theta, ul(k, 3), w); 

d21 dx2 - d21ambda dx2(theta, ul(k, j), w); 
nkj = rpeaks(k, j); 


value = nkj * log(delta x lambda) - delta + 
lambda + 
(d21 dx2 * (nkj - lambda * delta) / lambda - 
da * C eck c2) Cemix s 2) I) = 057 


if -isnan(value) 
y(k, j) = value; 
end 
end 


end 
end 


end 


function lcd ucl EgetapkecontEelmsiGve b OP) 


p = (1e-4:1e-4:1); 
ijs) e» Climestcior a, di c (epe c pl & WA) x19 ac Gl = 0)) of 
esla) / (B e s))e (og ./ (i = 1) e c9099))) = Se 
^ 8)5)5 


igi SS macia Es 043075) $ 
jut e eacha @ 0.025); 


well = ed 
Jie. => jul 
end 


nien 

m(end)); 

function certainty - get certainty curve(vK, mu, xK, chance prob) 
p = (1e-4:1e-4:1); 
E = mini(absi(p > chances prob); 


certainty = zeros(1, length(vK)); 


Eon J = TT ETSL 
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fp = cumtrapz(p, 1 ./ (sqrt(2 * pi * vK(j)) * p .* (1 - p 
yy) ae 


exp (((-1 


1i 
DI = x02) .^ 2595 
certainty(1, j) = 1 - fp(i); 


end 
end 


10.8 State-space Model with One Binary and Two 
Continuous Observations with a Circadian Input in the 
State Equation 


10.8.1 Experimental Data Example 


ndays = 5; 
T = 1440; 
N = ndays * T; 
t = (1:N); 


load('expm data one bin two cont circadian.mat'); 


Eee! se e ciel (x) 5 
Suele = mus) y 


i cox f peel ies 
Seam oce Sp 


base_prob = sum(n) / length(n); 


M = 2e6; 

ve = zeros(1, M); $ process noise variance 
rho - zeros(1, M); 
b0 = zeros(1, M); 
bl = zeros(1, M); 
r0 = zeros(1, M); 
ved, c memo, y 
Vig = Zeros), Mi); 
S0 = zeros(1, M); 
sl = zeros(1, M); 
vs = zeros(1, M); 


K = length(n) ; 


be joa = memos (il, 18) 5 
Ni7_joneterel = eros ll, 19) y 


ES lost c mon (Al, 19) y 
wlupdt = zeros (1, K); 
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x smth = zeros(1, 
w emea menos) (1, 


pEupdtE-ezeros» 


tol = 1e-8; % 
A = zeros(1, 


CW = zerosil, 
C= zeros (i. 


K) 
W = zeros i) Ki 
K 
K) 


ve(1) = 0.005; 
rho = 0.997 
b0(1) = 

loi (aL) = Uses 

xe (GL) = se), 
veal (1) = il; 
vr(1) = 0.005; 
EXO (al) = ms 
s1(1) = 1; 
vs(1) = 0.005; 
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KE 
Kg) § 


K) ; 


convergence criteria 


1 


) 8 


1 


log (base prob / (1 - base prob)); 


for k s PUR 
age (Ue ES 3b 
x pred(k) = x smth(1) + I(k); 
v pred(k) = ve(m) + ve(m); 
else 
x pred(k) = rho(m) « x updt(k - 1) + I(k); 
v_pred(k) = (rho(m) ^ 2) « v updt(k - 1) + ve(m); 
end 
C(k) = v pred(k) / (vr(m) « vs(m) + v pred(k) = ((r1(m) ^ 
2) « vs(m) + (sl(m) ^ 2) » vr(m))); 
upd t (la) = gets posterior mode Gar oreca) jae T O 
m), rl(m), b0(m), bi(m), vr(m), n(k), s(k), sO(m), sl(m), vs( 
m)); 
p updt(k) = 1 / (1 + exp((-1) « (bO(m) + bi(m) * x updt (k 
)23J 5 
y vrs cx 3b / (al X^ vrai) = (Ela ^ 2) 7 vel) 
+ ((s1(m ^ 2) / vs(m) + (bi(m) ^ 2) * p updt(k) * (1 - 
p updt (k))); 
end 
x smth(K) = x updt(K); 
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v smth(K) = v updt(K); 


W(K) = v smth(K) + (x smth(K) ^ 2); 
A(1:(end - 1)) = rho(m) « v updt(1:(end - 1)) ./ v pred (2:end 
); 
for k= UK = IN 
x smth(k) = x updt(k) + A(k) * (x_smth(k + 1) - x pred(k + 
DDE 
v_smth(k) = v updt(k) + (A(k) ^ 2) * (v smth(k + 1) - 


v_pred(k + 1)); 


CW(k) = A(k) * v smth(k + 1) + x smth(k) + x smth(k + 1); 
W(k) = v_smth(k) + (x_smth(k) ^ 2); 


if (m < M) 


rho(m + 1) = sum(CW) / sum(W(1:end - 1)); 
next ve = (sum(W(2:end)) + (rho(m + 1) ^ 2) x sum(W(1: ( 
end - 1))) - 2 x rho(m + 1) + sum(CW) - 
A e end) xc xvsmthi2isend) a a A a home Sr 
(Hasan) TET (Sal > 30))) 7)) «E 
(E c ^9) Y m 
ie (mextive = 0) $ check - in case this happens with 
experimental data 
ve(m + 1) = next ve; 
else 
ve(m + 1) = ve(m); 
end 


bb = fsolve(@(b) binary parameter derivatives(b, n, 
2 Eme, wp Ema), |l=S alll, opti meet (^ whlejolleny” » "xen )) y 


ale Maska = 0) $ check - in case this happens with 
experimental data 
DO (Gir ee 3D) = dele (Cae 
DMA) SAD) 
else 
b0(m + 1) = bO(m); 
b1(m + 1) = b1(m); 
end 
a = fminsearch(@(a) circadian parameters (a, rho(m + 1), 
PS Sima, Ey WO), ly Gemeet ERRA y “OE A 


It = caycan MD, 7); 


RE-Egetmimearspanametenssiousemth Wy Er e 
S = get linear parameters(x smth, W, s, K); 


next vr - get continuous variable variance update(r, R(1, 
3D). le, 3b); We 2 Emicia, 19) y 


212 10 MATLAB Code Examples 


next vs = get continuous variable variance update(s, S(1, 
1), (2, A), Win Ss fume 19) y 


if (abs(next vr - next vs) > 0.01) % overfitting control 
with experimental data 

rO(m + 1) = rO(m); 
rdum = d) = rim) 
omes RUHI 
sl(m + 1) = s1(m); 
vr(m + 1) = vr(m); 
vs(m + 1) = vs(m); 

else 
20 (Gm e 3L) e RL, ) y 
clin + 1) = Rie. Ls 
s0(m + 1) = S(1, 1); 
oltm c 1) e S, i) 
Wael SR 3D) = menda Wen 
vs(m + 1) = next vs; 

end 


mean dev = mean(abs([ve(m + 1) rho(m + 1) rO(m + 1) r1(m 
+ 1) vr(m + 1) sO(m + 1) s1(m + 1) vs(m + 1) b1(m + 1) bO(m + 
a) e 
[ve (m) rho(m) rO(m) r1(m) vr(m) sO(m) s1(m) vs(m) b1( 
m) b0(m)])); 


if mean dev « tol 
fprintt (m = <d\nx0 = sels \nver= <- T8fF\nrho = <. T8EN 
ninr0 = $.18fWMnrl = $.18f\nvr = %.18f\ns0 = %.18f\nsl = %.18f 
\nvs = %.18f\n\nbO = %.18f\nb1 = %.18f\n\n’, 
i, smell), Mm), dao), Os rn , seal (um) 2 1 y 
soim), sim, ven, bom bium 
fprintf('Converged at m = %d\n\n’, m); 
break; 
else 
fprintt (im = sd\nxO = SalSf\nvel= selSt\nrho = HIST 
ninr0 = %.18f\nrl = %.18f\nvr = %.18f\n\nsO = %.18f\ns1 = 


STE ENS = St vnm OS Sea nbl I wail m, 
x smth(1), ve(m + 1), rho(m + 1), rO(m + 1), r1(m 
+ 1), velm + 1), s00m 9 1), sitim e 1), velm * 1), boim + 1), 


bi(m  1)); 


eel c» aros (a, HQ E 
v pred = zeros(1, K); 


se wyle cs aos (al, 19) 7 
Whole = meros, 19) 7 


amen (seme!) c memos al, M6 = al) § $ x_smth(1) 
needed for next iteration 
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I 


X anda = Aaron (il, 19) 7 
p_updt = zeros(1, K); 


w= Zeros(i, Kj 
W = zeros(1, K) 
(GN c R EL 1 
(C. e rare. 1) 


end 

end 
end 
p smth = 1 ./ (1 + exp((-1) «+ (bO(m) + bi(m) « x smth))) ; 
r smth = (rO(m) + rl(m) * x smth) * std r; 
s smth = (sO(m) + sl(m) » x smth) * std s; 
index = (0:(K - 1)); 
t index = index / (60 « 24); 
ne alas = (HS T (=a) sO) g. 3 Bares ele 


bransp = 0.35 


subplot (611); 

hold on; 

¡loo (e imes, Way clor, MOS / 255) © (202 / 285)1)9 Gps 
yabe 

title('State Estimation with Experimental Data’); 

xlim([O t index(end)]); ylim([O (1.1 x max(y))1); 
set(gca,'xticklabel', []); 


subplot (612); 

n plot = NaN x* ones(1, K); 

i jala (ial e 0) = als 

GH last, im jollete, “itll”, “colo”, fil, 69 / 285, Ol, * 
markersize', 2); 

a (0) tlamdes (endo 2510 

set (gca, 'xticklabel', []); 

ylabel (UB) ns grid; 


subplot (613); 

hold on; 

plore fe ele: jo) ama, “E, Mireia, abe) p salaum( (0.98 e ima 
p smth)) (1.08 x max(p_smth))]); 

ylabel (MSP grid; 

xlim([0 t index(end)]); 

Seti(gca,’xticklabel’, (11) 


subplot (614); 


hold on; 

plc (ie amos, xe Simtel, Yoo, “calor”, 10 (5 (p, neviden , E 
¡loe (te else, ae v» Bel s, “cole”, 10 0.0 01) cesis 

xlim([0 t index(end)]); ylabel('(d) r_{k}'); 


set(gca,'xticklabel', []); 


subplot (615); 
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held on; 

[oleis (e sbaelsee, E media, "==", “eel”, [0.5 (25 / 285) (66 / 285)1, 
‘linewidth’ E 

los (E T er v» piel mn “cole”, [nb 0.5 (179 / 2858)1)9 cole 

xlim([0 t index(end)]); ylabel('(e) s (k]'); 

set(gca,'xticklabel', []); 


subplot(616); 


hold on; 

pole aumelzpx, Fx Gwe Ve’, imene y 35.55) x 

label 

xlim([0 t_index(end)]); ylim([(min(x smth) - 1) (max(x smth) + 1) 
DE 

grid; 

bestens (O90. 5:8 s 5) p dekle le AOL, 212007, (1000, "aba E 
OC OO AEA OO OE EON ONON '1200')); 


xlabel('time (24h clock)'); 


Pomerio y = Ger osten mee (Es reel, E, ©; 180, sel, 190, 1D, vi, 
y E, O, El, vE) 


M = 200; % maximum iterations 
it = zeros(1, M); 

f = zeros(1, M); 

df = zeros(1, M); 

aie, (1) = xe pere 


for 4 s I:(M = 1) 


i (ab) e alie (at) = ox pued = C a (Gell e we owe (eo x0 = teal y 
x pred) + sl x vr « (s - sO - sl « x pred) + 
Vr * VS x bl x (n - (1 / (1 + exp((-1) x (bO + bl + 
aie (3E) 99 33g 


ca) silos € o Wee we & ds * 2) 5 exp(b0 + bl + it(i)) 
i «i & eso c Toil oc ae) ^ 95 
eG + 1) = Seq) = EN) / eq 


if abs(it(i + 1) - it(i)) « le-14 
y = aie (al s» 1), 
return; 
end 
end 
error('Newton-Raphson failed to converge.'); 


end 


function y - binary parameter derivatives(b, n, x smth, v smth) 


zeros(1, 2); 
length (n); 


Y 
K 
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o 
o 
Il 

o 
E 


for k= LOK 

p(k) 1 / (1 + exp((-1) * (b0 + bti « x smth(k)))); 

(Gl) = we) +20) = 190) = 0.5 e Y sulle = (> 2) 3 
(bel x qb om 904) > (Gl = 2 3 199) 

yi(2) = y 2) + n(k) * x smth(k) - x smth(k) * p(k) - 0.5 * 
v smth(k) + bl * p(k) + (1 - p(k)) + (2 + x smth(k) + bl + 
(1 - 2 * p(k))); 
end 


end 
function y = get linear parameters(x smth, W, z, K) 


y = [K sum(x smth); sum(x smth) sum(W)] \ [sum(z); sum(z .x 
x smth)]; 


end 


function y = get continuous variable variance update(z, r0, rl, W 
ETIE HES 
y= laea ako GO 2j c Gal o E e suo 
= 20:06 un(a) = A sal lola (xe mica, A) Er? 20 
* rl « sum(x smth)) / K; 


end 
function y - circadian parameters(a, rho, x smth, t, T) 
- rhythm(a, T, t) 


Wi i 
ya (E 24) = 2 c (laser) s rx lE) RHS Sate ss 2) lao (Qu 
(asen) x» z Emilia lala (sei: e 1%) 7 


end 

Erica y = aeyAElann(cy, AP, 18) % the a0 is ignored 
VEO aa) cual O o ee E) eb El) a Cosa y aL t 
T) + 


2 Ho sin(2@ 4 pi xc te TUA a5) = cosi(2 x pa Et 
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Chapter 11 Y 
List of Supplementary MATLAB om 
Functions 


All the MATLAB code examples accompanying this book can be run directly. The 
examples are self-contained and do not require additional path variables being set 
up. The following is a partial list of the supplementary MATLAB functions that are 
called at various stages by the state estimators. 


get linear parameters (. . .) 

Calculates the updates for the constant coefficients (e.g., yo and yı) for a 
continuous variable (e.g., rg). If this function is present in a MATLAB example 
where there is an MPP, but not a continuous variable, then it calculates the 
constant coefficients based on the MPP amplitudes. 
get maximum variance (...) or get continuous variable variance update (. . .) 

Calculates the sensor noise variance update (e.g., 0?) for a continuous variable 
(e.g., rg). If this function is present in a MATLAB example where there is an 
MPP, but not a continuous variable, then it calculates the sensor noise variance 
based on the MPP amplitudes. 
get_linear_parameters_for_mpp (...) 

Calculates the updates for the constant coefficients (e.g., yo and y1) for a 
series of MPP amplitudes (e.g., rx). This function is used to calculate the updates 
corresponding to an MPP when a continuous variable is also present. 
get_maximum_variance_for_mpp (...) 

Calculates the sensor noise variance update (e.g., 0?) for a series of MPP 
amplitudes. This function is used to calculate the update corresponding to an 
MPP when a continuous variable is also present. 
get posterior mode (...) or get state update (...) 

Calculates the update x; based on the Newton-Raphson method 
get pk conf lims (...) 

Calculates the confidence limits for the probability of binary event occurrence 
Pk 
get certainty curve (...) 
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Calculates the HAI value based on the probability of binary event occurrence 
px exceeding a baseline value 
* rhythm (...) 
Calculates the cortisol-related circadian term 7; in the state equation 
* circadian parameters (...) 
Calculates the log-likelihood term to be optimized when estimating the 
(cortisol-related) circadian rhythm terms in the state equation 
e get log likelihood (...) 
Calculates the log-likelihood of the term involving the CIF 
* get ks plot(...) 
Calculates the Kolmogorov-Smirnov (KS) plot for assessing the goodness of 
fit of a CIF to point process observations 
* Other functions related to a CIF 
Functions such as fetch lambda (...), dlambda dx (...), £(...), and mu (...) 
are all supplementary functions that calculate various components or derivatives 
related to an HDIG-based CIF 
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